Real-time Iot device reliability and maintenance system and method

ABSTRACT

The present invention generally relates to systems and methods for detecting and/or isolating any causes of defective and/or partially defective IoT device or individual sensor device(s). In embodiments the present invention generally relates to fixing, replacing, and/or troubleshooting IoT devices and/or individual sensor device(s) that are defective and/or partially defective.

REFERENCE TO OTHER APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalApplication No. 62/905,670, filed on Sep. 25, 2019 and entitled“REAL-TIME EVENT TRANSCRIPTION SYSTEM AND METHOD” the contents of whichis hereby incorporated by reference herein in its entirety.

This application is related to U.S. patent application Ser. No.16/527,743, filed on Jul. 31, 2019 and entitled “REAL-TIME EVENTTRANSCRIPTION SYSTEM AND METHOD” which is a continuation-in-part of U.S.patent application Ser. No. 16/379,368, filed on Apr. 9, 2019 andentitled “REAL-TIME EVENT TRANSCRIPTION SYSTEM AND METHOD” which is acontinuation-in-part of U.S. patent application Ser. No. 16/220,394,filed on Dec. 14, 2018 and entitled “REAL-TIME EVENT TRANSCRIPTIONSYSTEM AND METHOD,” which is a continuation of U.S. patent applicationSer. No. 15/865,928, filed on Jan. 9, 2018 and entitled “REAL-TIME EVENTTRANSCRIPTION SYSTEM AND METHOD,” which is a continuation of U.S. patentapplication Ser. No. 14/214,894, filed on Mar. 15, 2014 and entitled“REAL-TIME EVENT TRANSCRIPTION SYSTEM AND METHOD,” which claims thebenefit of and priority to U.S. Provisional Patent Application No.61/798,710, filed on Mar. 15, 2013, and U.S. Provisional PatentApplication No. 61/802,463, filed on Mar. 16, 2013, the contents of allof which are incorporated by reference herein in their entirety.

This application is also related to co-pending U.S. patent applicationSer. No. 16/533,312, filed on Aug. 6, 2019 and entitled “REAL-TIME EVENTTRANSCRIPTION SYSTEM AND METHOD” which is a continuation-in-part of U.S.patent application Ser. No. 16/379,368, filed on Apr. 9, 2019 andentitled “REAL-TIME EVENT TRANSCRIPTION SYSTEM AND METHOD,” the contentsof which is incorporated by reference herein in its entirety.

This application is also related to co-pending U.S. patent applicationSer. No. 16/527,743, filed on Jul. 31, 2019 and entitled “REAL-TIMEEVENT TRANSCRIPTION SYSTEM AND METHOD” which is a continuation-in-partof U.S. patent application Ser. No. 16/379,368, filed on Apr. 9, 2019and entitled “REAL-TIME EVENT TRANSCRIPTION SYSTEM AND METHOD,” thecontents of which is incorporated by reference herein in its entirety.

FIELD

The present invention generally relates to systems and methods fordetecting and/or isolating any causes of defective and/or partiallydefective “Internet of Things” (IoT) device or individual sensordevice(s). In embodiments the present invention generally relates tofixing, replacing, and/or troubleshooting IoT devices and/or individualsensor device(s) that are defective and/or partially defective.

BACKGROUND

The expansion and utilization of IoT devices has led to the deploymentof various devices that use or include various types of sensor devicesthat may gather and communicate data in a variety of environments,including in homes (e.g. connected consumer devices), offices andbuildings (e.g. connected enterprise devices), and/or manufacturingplants (e.g. connected industrial devices), to name a few. Theeffectiveness of IoT devices is dependent on the reliability of the dataobtained and generated using the IoT devices—highlighting the importanceof detecting and/or isolating any causes of defective and/or partiallydefective sensor devices. Current methods and processes for identifyingdefective or malfunctioning devices or individual sensors used thereinare often unreliable, require the sensor device to be powered off, anddo not account for multiple devices failing simultaneously. Thus, itwould be beneficial to provide systems and methods for detecting and/orisolating any causes of defective and/or partially defective IoT deviceor individual sensor device. It would be further beneficial to providesystems and methods for curing causes of defective and/or partiallydefective sensor devices.

SUMMARY

The present invention generally relates to an improved real-time eventtranscription system and method.

In embodiments, a method may comprise: (a) obtaining, by a sensor moduleon a computer system, first sensor feedback information from each sensordevice of a first group of sensor devices of a plurality of sensordevices operatively connected via a network to the computer system,wherein the first sensor feedback information includes: (1)identification information associated with each respective sensor deviceproviding first sensor feedback information; (2) respective firstreadout information provided by each respective sensor device; (3)respective first metadata information associated with and provided byeach respective sensor device; and (4) respective first timestampinformation indicating a respective first time at which the respectivefirst readout information was generated by the respective sensor device;(b) generating, by the sensor module, respective second timestampinformation indicating a respective second time at which the firstsensor feedback information was obtained by the sensor module; (c)storing, in at least one memory element of memory operatively connectedto the computer system, the first sensor feedback information, includingthe second timestamp, wherein the at least one memory element furtherincludes historical sensor feedback information associated with one ormore sensor devices of the plurality of sensor devices, wherein thehistorical sensor feedback information includes: (1) identificationinformation associated with each respective sensor device of the one ormore sensor devices associated with the historical sensor feedbackinformation; (2) respective historical readout information provided byeach respective sensor device of the one or more sensor devicesassociated with the historical sensor feedback information; (3)respective historical metadata information associated with and providedby each respective sensor device of the one or more sensor devicesassociated with the historical sensor feedback information; and (4)respective historical timestamp information indicating a respectivehistorical time at which the respective historical feedback informationwas generated by the respective sensor device of the one or more sensordevices associated with the historical sensor feedback information,wherein the memory comprises: (i) a reliability rating databaseoperatively connected to the computer system, which comprises arespective historical reliability rating for each sensor device of theplurality of sensor devices, and (ii) a sensor information databasewhich includes for each sensor device of the plurality of sensordevices, sensor information including: (a) sensor identificationinformation unique to each sensor device; (b) location informationindicating a sensor location associated with each sensor device; and (C)data range information indicating a range of permissible sensor valuesfor the respective sensor device; (d) calculating, by the sensor module,for each sensor device of the first group of sensor devices, respectivesensor variance information based on at least one of the following: (1)respective first sensor feedback information; (2) respective historicalsensor feedback information; and (3) respective predictive data, whereinthe respective predictive data is generated by the computer system bypopulating a database with respective exemplary readout informationassociated with a sensor device of the first group of sensor devices;(e) calculating, by the sensor module for each sensor device of thefirst group of sensor devices, respective sensor deviation informationbased on at least one of the following: (1) respective sensor varianceinformation; and (2) first sensor feedback information; (f) storing, bythe sensor module in the at least one memory element, the respectivesensor variance information and the respective sensor deviationinformation associated with each sensor device of the first group ofsensor devices; (g) determining, by the computer system, a reliabilityrating of each sensor device of the first group of sensor devices basedon at least the following: the respective sensor variance information;the respective sensor deviation information; and the respective firstsensor feedback information; (h) determining, by the computer system,whether at least one sensor device of the first group of sensor devicesis out of specification by performing at least the following steps: (1)comparing, by the computer system, first respective sensor feedbackinformation associated with the at least one sensor device with firstrespective data range information associated with the at least one othersensor device, wherein, the computer system determines that the at leastone sensor device is out of specification when the first respectivesensor feedback data varies by more than a first predetermined thresholdfrom the first respective data range information; (2) comparing, by thecomputer system, the first respective sensor feedback information withsecond respective sensor feedback information associated with one ormore other sensor devices of the first group of sensor devices, whereinthe computer system determines that the at least one sensor device isout of specification when the first respective sensor feedback data isinconsistent with the second respective sensor feedback data; and (3)determining that the at least one sensor device is out of specificationby performing the following steps: (A) obtaining respective reliabilityratings for at least the at least one sensor device; (B) in the eventthe respective reliability ratings indicate the at least one sensor isunreliable, the at least one sensor device is determined to be out ofspecification; and (C) in the event the respective reliability ratingsindicate the at least one sensor is not unreliable, the at least onesensor device is not determined to be out of specification; (i) in thecase where at least one sensor device is determined to be out ofspecification, performing the following steps: (1) generating, by thecomputer system, a notice message identifying the at least one sensordevice that is out of sequence, where the notice message includes causeinformation indicating the at least one sensor device is out ofspecification; and (2) sending the notice message to a device associatedwith at least one user of the computer system; and (j) updating, by thesensor module, the historical sensor feedback information with the firstsensor feedback information.

In embodiments, in step (d), the sensor module calculates, for eachsensor device of the first group of sensor devices, respective sensorvariance information further based on: calculated results of a HiddenMarkov Model (HMM) algorithm utilizing one or more of the following asan input: the respective sensor variance information; the respectivesensor deviation information; the respective first sensor feedbackinformation; and respective historical sensor feedback information;

In embodiments, in step (d), the sensor module calculates, for eachsensor device of the first group of sensor devices, respective sensordeviation information further based on at least one of the following:(3) respective historical sensor feedback information; (4) respectivepredictive data; and calculated results of a Hidden Markov Model (HMM)algorithm utilizing one or more of the following as an input: therespective sensor variance information; the respective sensor deviationinformation; the respective first sensor feedback information; andrespective historical sensor feedback information;

In embodiments, in step (h)(2) of comparing the first respective sensorfeedback information with second respective sensor feedback informationfurther comprises comparing the first respective sensor feedbackinformation with one or more of the following: (A) historical sensorfeedback information associated with the respective sensor device; (B)historical sensor feedback information associated with the one or moreother sensor devices of the first group of sensor devices; (C)predictive data associated with the respective sensor device, whereinthe respective predictive data is generated by the computer systemutilizing respective exemplary readout information associated with therespective sensor; (D) predictive data associated with the one or moreother sensor devices of the first group of sensor devices, wherein therespective predictive data is generated by the computer system utilizingrespective exemplary readout information associated with the one or moreother sensor devices of the first group of sensor devices;

In embodiments, the computer system weights respective sensor feedbackinformation associated with one or more sensor devices of the pluralityof sensor devices that are associated with a first respectivereliability rating indicating that one or more sensor devices arereliable more than respective sensor feedback information associatedwith one or more sensor devices of the plurality of sensor devices thatare associated with a second respective reliability rating indicatingthat one or more sensor devices are unreliable.

In embodiments, determining whether the at least one sensor device ofthe first group of sensor devices is out of sequence further comprises:(4) obtaining, by the computer system from the reliability ratingdatabase, a respective historical reliability rating for each sensordevice of the first group of sensor devices; and (5) weightingrespective sensor feedback information associated with reliable sensordevices of the first group of sensor devices more than sensor feedbackinformation associated with unreliable sensor devices of the first groupof sensor devices. In embodiments, in the event that both the respectivehistorical reliability rating and the determined reliability ratingassociated with the at least one sensor device indicate that the atleast one sensor device is unreliable, the at least one sensor device isdetermined to be out of sequence. In embodiments, in the event that therespective historical reliability rating indicates that the at least onesensor device is reliable and the determined reliability ratingindicates the at least one sensor device is unreliable, the respectivesensor device is determined to be out of sequence. In embodiments, inthe event that the respective historical reliability rating indicatesthe at least one sensor device is unreliable and the determinedreliability rating indicates the at least one sensor device is reliable,the at least one sensor device is determined to be not out of sequence.

In embodiments, the method further comprises: (k) identifying, by thecomputer system, respective first readout information associated withone or more sensors that are determined to be reliable based on thedetermined reliability rating; (l) generating, by the computer system,first machine-readable instructions to provide a first graphical userinterface, wherein the first graphical user interface presents a displayof the respective first readout information associated with the one ormore sensors that are determined to be reliable; and (m) transmitting,by the computer system to an administrator device associated with anadministrator of the computer system, the first machine-readableinstructions, wherein, upon receipt, the administrator device executesthe first machine-readable instructions such that the first graphicaluser interface is displayed on a display screen associated with theadministrator device.

In embodiments, the method further comprises: (k) obtaining, by thecomputer system, respective sensor information associated with the atleast one sensor device that is out of sequence; (l) generating, by thecomputer system, a report comprising at least the following: (1) therespective sensor information; and (2) a repair message indicating thatthe at least one sensor device is out of sequence; and (m) sending, bythe computer system to the device associated with the at least one user,the report.\

In embodiments, the method further comprises: (k) obtaining, by thecomputer system, respective sensor information associated with the atleast one sensor device that is out of sequence; (l) determining, by thecomputer system based on the respective sensor information, at least oneitem associated to purchase, wherein the at least one item is associatedwith the at least one sensor device; (m) generating, by the computersystem, a purchase order comprising at least the following: (1) therespective sensor information associated with the at least one sensordevice that is out of sequence; and (2) the at least one item; and (n)sending, by the computer system, the purchase order to a third partyvendor. In embodiments, the method further comprises: (o) generating, bythe computer system, a report comprising at least the following: (1) therespective sensor information; (2) information identifying the at leastone item; and (3) a repair message indicating at least the following:(i) that the at least one sensor device is out of sequence; and (ii) theat least one item; and (p) sending, by the computer system to the deviceassociated with the at least one user, the report. In embodiments, themethod further comprises: (q) sending, by the computer system to anelectronic device operatively connected to the computer system, therepair message.

In embodiments, the method further comprises: (k) obtaining, by thecomputer system, respective sensor identification information associatedwith the at least one sensor device that is out of sequence; (l)storing, in a flagged sensor database of the computer system, therespective sensor identification information; (m) generating, by thecomputer system, second machine-readable instructions to: (1) flag, bythe flagged sensor database, obtained respective sensor feedback dataassociated with the sensor identification information stored in theflagged sensor database; and (2) store, in the flagged sensor database,the flagged respective sensor feedback data; and (n) executing, by thecomputer system, the second machine-readable instructions. Inembodiments, the method further comprises: (o) generating, by thecomputer system, third machine-readable instructions to: (1) ignore datareceived from the at least one sensor device that is out of sequence;and (2) cease transmitting data to the at least one sensor device thatis out of sequence; and (p) sending, by the computer system to eachsensor device of a second group of sensor devices, the thirdmachine-readable instructions, wherein the second group of sensordevices is the first group of sensor devices less the at least onesensor device, and wherein, upon receipt of the third machine-readableinstructions, each of the second group of sensor devices executes thethird machine-readable instructions.

In embodiments, the method further comprises: (k) obtaining, by thecomputer system for each sensor device of the first group of sensordevices, first weather information associated with at least a firstrespective temperature at the first time within a predetermined radiusof respective location information, wherein the sensor varianceinformation is further based on the first weather information, andwherein the sensor deviation information is further based on the firstweather information.

In embodiments, wherein the first group of sensor devices selected bythe sensor module by performing the following steps: (1) determining, bythe sensor module for each sensor device of the plurality of sensordevices, respective location information; (2) selecting, by the sensormodule, a group of sensor devices of the plurality of sensor devicesbased on the respective location information, wherein each sensor deviceof the group of sensor devices are within a predetermined radius; and(3) determining, by the sensor module for each sensor device of thegroup of sensor devices, a respective sensor type based on respectivesensor identification information, wherein the first group of sensordevices is each sensor device of the group of sensor devices that are ofa first type of sensor device.

In embodiments, the method further comprises: (k) determining, by thecomputer system, whether a first sensor device of the first group ofsensor devices is detecting an emergency, based on at least thefollowing: (1) comparing, by the computer system, first sensor feedbackinformation associated with the first sensor device with first datarange information associated with the first sensor device, wherein thecomputer system determines that the first sensor feedback informationvaries by more than a second predetermined threshold from the first datarange information; (2) comparing, by the computer system, the firstsensor feedback information with at least second sensor feedbackinformation associated with a second sensor device of the first group ofsensor devices, wherein the computer system determines that the firstsensor feedback data is consistent with at least the second sensorfeedback data; and (3) analyzing, by the computer system a firstdetermined reliability rating associated with the first sensor device,wherein the first determined reliability rating indicates that the firstsensor device is reliable; and (l) generating, by the computer system,an emergency notification indicating that the first sensor device isdetecting an emergency event; and (m) sending the emergency notificationto the at least one administrator of the computer system. Inembodiments, the method further comprises: (n) sending, by the computersystem to an electronic device operatively connected to emergencyservices, the emergency notification.

In embodiments, a process for detecting at least a partially defectivesensor device may include: (a) providing a plurality of sensor devicesoperatively connected via a network to a computing device; (b) providinga reliability rating database operatively connected to the computingdevice, wherein the reliability rating database comprises a respectivehistorical reliability rating for each sensor device of the plurality ofsensor devices; (c) storing, by the computing device, in at least onememory element operatively connected to the computer device, sensorinformation associated with each sensor device of the plurality ofsensor devices, wherein the sensor information includes, for each sensordevice of the plurality of sensor devices: (1) sensor identificationinformation unique to each sensor device; (2) location informationindicating sensor location; (3) data range information indicating arange of sensor values, the data range information including: (i) amaximum sensor value; and (ii) a minimum sensor value; (d) accessing, bya sensor module of the computing device, the sensor information; (e)selecting, by the sensor module, at least a first group of sensordevices of the plurality of sensor devices based on at least the sensorinformation, (f) obtaining, by the sensor module, first sensor feedbackinformation from one or more sensor devices of the first group of sensordevices, wherein the sensor feedback information includes: (1)identification information associated with a respective sensor deviceproviding the first sensor feedback information; (2) first feedback dataprovided by the respective sensor device in response to stimuli; and (3)a first timestamp indicating a first time at which the first feedbackdata was generated by the respective sensor device; (g) generating, bythe sensor module, a second timestamp indicating a time at which thefirst sensor feedback data was obtained by the sensor module; (h)storing, by the sensor module, the first sensor feedback information,including the second timestamp in the at least one memory element; (i)generating, by the sensor module, for each sensor device of the firstgroup of sensor devices, sensor variance information based on at leastthe first sensor feedback information; (j) generating, by the sensormodule for each sensor device of the first group of sensor devices,sensor deviation information based on at least the first sensor feedbackinformation and the generated sensor variance information; (k) storing,by the sensor module in the at least one memory element, respectivesensor variance information and respective sensor deviation informationassociated with each sensor device of the first group of sensor devices;(l) determining, by the computing device, a determined reliabilityrating of each sensor device of the first group of sensor devices basedon at least the following: (1) respective sensor variance information;(2) respective sensor deviation information; and (3) respective firstsensor feedback information; (m) determining, by the computing device,whether at least one sensor device of the first group of sensor devicesis at least partially defective based on at least the following: (1)comparing, by the computing device, first respective sensor feedbackinformation associated with the at least one sensor device with firstrespective data range information associated with the at least one othersensor device; (2) comparing, by the computing device, the firstrespective sensor feedback information with second respective sensorfeedback information associated with one or more other sensor devices ofthe first group of sensor devices; and (3) analyzing respectivedetermined reliability rating, wherein the computing device determinesthat the at least one sensor device is at least partially defective whenat least one of the following occurs: (A) the computing devicedetermines that the first respective sensor feedback data varies by morethan a first predetermined threshold from the first respective datarange information; (B) the computing device determines that the firstrespective sensor feedback data is inconsistent with the secondrespective sensor feedback data; and (C) the computing device determinesthat the first respective reliability rating indicates the at least onesensor device is unreliable; (n) generating, by the computing device, anotice message identifying the at least one sensor device that is atleast partially defective, where the notice message includes causeinformation indicating the at least one sensor device is at leastpartially defective; and (o) sending the notice message to a deviceassociated with at least one user of the computing device.

In embodiments, the computing device weighs respective sensor feedbackinformation associated with one or more sensor devices of the pluralityof sensor devices that are associated with a first respectivereliability rating indicating that one or more sensor devices arereliable more than respective sensor feedback information associatedwith one or more sensor devices of the plurality of sensor devices thatare associated with a second respective reliability rating indicatingthat one or more sensor devices are unreliable.

In embodiments, determining whether the at least one sensor device ispartially defective may further comprise: (4) obtaining, by thecomputing device from the reliability rating database, respectivehistorical reliability ratings for each sensor device of the first groupof sensor devices. In embodiments, in the event that both the obtainedreliability rating and the determined reliability rating indicate therespective sensor device is unreliable, the respective sensor device isat least partially defective. In embodiments, in the event that both theobtained reliability rating indicates the respective sensor device isreliable and the determined reliability rating indicates the respectivesensor device is unreliable, the respective sensor device is not atleast partially defective. In embodiments, in the event that both theobtained reliability rating indicates the respective sensor device isunreliable and the determined reliability rating indicates therespective sensor device is reliable, the respective sensor device isnot at least partially defective.

In embodiments, the method further comprises: (p) identifying, by thecomputing device, respective feedback data associated with one or moresensors that are rated as reliable; (q) generating, by the computingdevice, first machine-readable instructions to provide a first graphicaluser interface, wherein the first graphical user interface presents adisplay the respective feedback data associated with the one or moresensors that are rated as reliable; and (r) presenting, by the computingdevice based on the first machine-readable instructions, the firstgraphical user interface on a display screen associated with thecomputing device.

In embodiments, the method further comprises: (p) generating, by thecomputing device, second machine-readable instructions includingtroubleshoot instructions, wherein the troubleshoot instructions areassociated with the at least one sensor device; (q) sending, by thesensor module to the at least one sensor device via the network, thesecond machine-readable instructions, wherein, upon receipt of thesecond machine-readable instructions, the at least one sensor deviceperforms a troubleshoot operation by executing the secondmachine-readable instructions; (r) obtaining, by the sensor module fromthe at least one sensor device, second sensor feedback information; (s)generating, by the sensor module, a third timestamp indicating a time atwhich the second sensor feedback information was obtained by the sensormodule; (t) generating, by the sensor module, for the at least onesensor device, updated sensor variance information based on at least thesecond sensor feedback information and the sensor feedback information;(u) generating, by the sensor module for the at least one sensor device,updated sensor deviation information based on at least the sensorfeedback information, the generated sensor variance information, thesecond sensor feedback information, the updated sensor varianceinformation and the sensor deviation information; (v) determining, bythe computing device, whether at least one sensor device of the firstgroup of sensor devices is at least partially defective based on atleast the following: (1) comparing, by the computing device, firstrespective second sensor feedback information associated with the atleast one sensor device with the first respective data range informationassociated with the at least one other sensor device; (2) comparing, bythe computing device, the first respective second sensor feedbackinformation with second respective second sensor feedback informationassociated with one or more other sensor devices of the first group ofsensor devices; and (3) analyzing at least the updated sensor varianceinformation and the updated sensor deviation information, wherein thecomputing device determines that the at least one sensor device is atleast partially defective when at least one of the following occurs: (A)the computing device determines that the first respective second sensorfeedback information varies by more than the first predeterminedthreshold from the first respective data range information; (B) thecomputing device determines that first respective second sensor feedbackinformation is inconsistent with the second respective second sensorsecond feedback information; and (C) the computing device determinesthat the sensor variance information and the updated sensor deviationinformation indicate the at least one sensor device is unreliable; (w)generating, by the computing device, an updated notice messageidentifying whether the at least one sensor device is at least partiallydefective; and (x) sending the updated notice message to at least oneadministrator of the computing device.

In embodiments, the method further comprises: (p) obtaining, by thecomputing device, respective sensor information associated with the atleast one sensor device that is at least partially defective; (q)generating, by the computing device, a report comprising at least thefollowing: (1) the respective sensor information; and (2) a repairmessage indicating at least that the at least one sensor device is atleast partially defective; and (r) sending, by the computing device tothe device associated with the at least one user, the report. Inembodiments, the method further comprises: (s) sending, by the computingdevice to an electronic device operatively connected to the computingdevice, the report.

In embodiments, the method further comprises: (p) obtaining, by thecomputing device, respective sensor information associated with the atleast one sensor device that is at least partially defective; (q)determining, by the computing device based on the sensor information, atleast one item to purchase, wherein the one item is associated with theat least one sensor device; (r) generating, by the computing device, apurchase order comprising at least the following: (1) the respectivesensor information associated with the at least one sensor device thatis at least partially defective; and (2) the at least one item; and (s)sending, by the computing device, the purchase order to a third partyvendor. In embodiments, the method further comprises: (t) generating, bythe computing device, a report comprising at least the following: (1)the respective sensor information; (2) information identifying the atleast one item; and (3) a repair message indicating at least thefollowing: (i) that the at least one sensor device is at least partiallydefective; (ii) the at least one item; (u) sending, by the computingdevice to the device associated with the at least one user, the report.In embodiments, the method further comprises: (v) sending, by thecomputing device to an electronic device operatively connected to thecomputing device, the repair message.

In embodiments, the method further comprises: (p) obtaining, by thecomputing device, respective sensor identification informationassociated with the at least one sensor device that is at leastpartially defective; (q) storing, in a flagged sensor database of thecomputing device, the respective sensor identification information; (r)generating, by the computing device, second machine-readableinstructions to: (1) flag, by the flagged sensor database, obtainedrespective sensor feedback data associated with the sensoridentification information stored in the flagged sensor database; and(2) store, in the flagged sensor database, the flagged respective sensorfeedback data; and (s) executing, by the computing device, the secondmachine-readable instructions. In embodiments, the flagged respectivesensor feedback data is stored with the respective sensor identificationinformation. In embodiments, the method further comprises: (t)generating, by the computing device, third machine-readable instructionsto: (1) ignore data received from the at least one sensor device that isat least partially defective; and (2) cease transmitting data to the atleast one sensor device that is at least partially defective; and (u)sending, by the computing device to each sensor device of a second groupof sensor devices, the third machine-readable instructions, wherein thesecond group of sensor devices is the first group of sensor devices lessthe at least one sensor device, and wherein, upon receipt of the thirdmachine-readable instructions, each of the second group of sensordevices executes the third machine-readable instructions.

In embodiments, the method further comprises: (p) generating, by thecomputing device, a reliability report comprising, for each sensordevice of the first group of sensor devices, at least: (1) a respectivereliability rating; and (2) respective feedback data; and (q)transmitting, by the computing device to the device associated with theat least one user, the reliability report.

In embodiments, the method further comprises: (p) obtaining, by thecomputing device for each sensor device of the first group of sensordevices, first weather information associated with at least a firstrespective temperature at the first time within a predetermined radiusof respective location information, wherein the sensor varianceinformation is further based on the first weather information, andwherein the sensor deviation information is further based on the firstweather information.

In embodiments, the first group of sensor devices is selected based onlocation information.

In embodiments, the first group of sensor devices is selected based onthe sensor identification information.

In embodiments, the location information, for each sensor device of theplurality of sensor devices, includes respective hierarchicalinformation associated with a respective sensor device. In embodiments,the first group of sensor devices is selected based at least in part onrespective hierarchical information.

In embodiments, the first group of sensor devices selected by the sensormodule by performing the following steps: (1) determining, by the sensormodule for each sensor device of the plurality of sensor devices,respective location information; (2) selecting, by the sensor module, agroup of sensor devices of the plurality of sensor devices based on therespective location information, wherein each sensor device of the groupof sensor devices are within a predetermined radius; and (3)determining, by the sensor module for each sensor device of the group ofsensor devices, a respective sensor type based on respective sensoridentification information, wherein the first group of sensor devices iseach sensor device of the group of sensor devices that are of a firsttype of sensor device.

In embodiments, the senor identification information comprises, for eachsensor device, at least the following: (A) a sensor type; (B) sensorspecification information; and (C) sensor running time informationindicating a respective amount of time a respective sensor has beenoperating.

In embodiments, the sensor information further includes: (4) sensorproximity information, indicating one or more sensor devices within apredefined distance of a respective sensor device.

In embodiments, determining whether the at least one sensor device is atleast partially defective is further based at least partially onrespective sensor variance information, respective first timestamp, andrespective second timestamp.

In embodiments, the plurality of sensor devices includes at least oneof: (1) a temperature sensor; (2) a pressure sensor; (3) a torquesensor; (4) a MEMS sensor; (5) a humidity sensor; (6) an air moisturesensor; (7) an air flow sensor; (8) a dielectric soil moisture sensor;(9) an optical sensor; (10) an electro-chemical sensor; (11) anaccelerometer; (12) a gyrometer; (13) a magnetometer; (14) a proximitysensor; (15) an air bubble detector (16) a piezo film sensor; (17) anangle or position sensor; (18) a voltage sensor; (19) a current sensor;(20) a magnetic or electrical field sensor; (21) an audio sensor; (22) apH sensor; (23) a time sensor; (24) a biological sensor; (25) abiometric sensor and (26) a radiation sensor.

In embodiments, the method further comprises: (p) determining, by thecomputing device, whether a first sensor device of the first group ofsensor devices is detecting an emergency, based on at least thefollowing: (1) comparing, by the computing device, first sensor feedbackinformation associated with the first sensor device with first datarange information associated with the first sensor device, wherein thecomputing device determines that the first sensor feedback informationvaries by more than a second predetermined threshold from the first datarange information; (2) comparing, by the computing device, the firstsensor feedback information with at least second sensor feedbackinformation associated with a second sensor device of the first group ofsensor devices, wherein the computing device determines that the firstsensor feedback data is consistent with at least the second sensorfeedback data; and (3) analyzing, by the computing device a firstdetermined reliability rating associated with the first sensor device,wherein the first determined reliability rating indicates that the firstsensor device is reliable; and (q) generating, by the computing device,an emergency notification indicating that the first sensor device isdetecting an emergency event; and (r) sending the emergency notificationto the at least one administrator of the computing device.

In embodiments, the method further comprises: (p) sending, by thecomputing device to an electronic device operatively connected to thecomputing device, the notice message.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described withreferences to the below referenced accompanying figures.

FIG. 1 illustrates an initial set of connections between a plurality ofexemplary entity types according to one or more embodiments of theinvention.

FIG. 2 illustrates an example of a state transition diagram according toone or more embodiments of the invention.

FIG. 3 demonstrates an illustrative process of applying a Hidden MarkovModel transformation according to one or more embodiments of theinvention.

FIG. 4 illustrates paths of reaching a final observation through astate-transition analysis based on past impressions and futurepredictions according to one or more embodiments of the invention.

FIG. 5 depicts a potential sequence of event elements within an eventaccording to one or more embodiments of the invention.

FIG. 6 depicts a further potential sequence of event elements within anevent according to one or more embodiments of the invention.

FIG. 7 shows steps for processing and transforming inputs in order togenerate output matrices according to one or more embodiments of theinvention.

FIG. 8 shows steps for processing and transforming inputs in order toreturn a user prediction matrix according to one or more embodiments ofthe invention.

FIG. 9 illustrates steps for processing and transforming inputs in orderto return a User Prediction State Matrix according to one or moreembodiments of the invention.

FIG. 10 depicts steps for processing and transforming inputs in order toreturn an Observation Matrix according to one or more embodiments of theinvention.

FIG. 11 shows steps for processing and transforming inputs in order toreturn Analyzed Responses according to one or more embodiments of theinvention.

FIG. 12 illustrates steps for processing and transforming inputs inorder to return User Queries according to one or more embodiments of theinvention.

FIG. 13 shows steps for processing and transforming inputs in order toreturn an Affinity Matrix according to one or more embodiments of theinvention.

FIG. 14 illustrates a series of ongoing processing steps for anoperating or running event prediction system and method in accordancewith one or more embodiments of the invention.

FIG. 15 shows steps for processing and transforming inputs in order toreturn a set of Selected States, according to one or more embodiments ofthe invention.

FIG. 16 shows steps for processing and transforming inputs in order toreturn an Affinity Matrix, according to one or more embodiments of theinvention.

FIG. 17 shows steps for processing and transforming inputs in order toreturn a credibility score for a plurality of devices, wherein thecredibility score is used to evaluate, detect and optionally mitigatemalformed data attempts, according to one or more embodiments of theinvention.

FIG. 18 illustrates various aspects of a networked computing systemaccording to one or more embodiments of the invention.

FIG. 19 illustrates various aspects of a communicating network ofcomputing server and a plurality of computing device clients accordingto one or more embodiments of the invention.

FIG. 20 illustrates a presentation of a screenshot of a graphical userinterface on a portable client computing device according to one or moreembodiments of the invention.

FIG. 21 depicts a further presentation of a screen of a graphical userinterface on a portable client computing device according to one or moreembodiments of the invention.

FIG. 22 is a flow chart of a process for generating an accurate reportin accordance with an exemplary embodiment of the present invention.

FIG. 23 is a flow chart of a process for determining accuracy of areport in accordance with an exemplary embodiment of the presentinvention.

FIG. 24 is a flow chart of a process for predicting conditions inaccordance with an exemplary embodiment of the present invention.

FIGS. 25A and 25B are flow charts of a process for predicting marketconditions in accordance with exemplary embodiments of the presentinvention.

FIG. 26 is a flow chart of a process for gathering information inaccordance with an exemplary embodiment of the present invention.

FIG. 27 is an exemplary block diagram illustrating a computing devicegenerating and transmitting an event stimulus message and a new stimulusmessage to a first group of users in accordance with exemplaryembodiments of the present invention.

FIG. 28 is an exemplary block diagram illustrating a computing devicereceiving a first response from a device in accordance with exemplaryembodiments of the present invention.

FIG. 29 is an exemplary block diagram illustrating a computing devicereceiving a first response from a device in accordance with exemplaryembodiments of the present invention.

FIG. 30 is an exemplary block diagram illustrating a computing devicegenerating and transmitting a report and a reliability report to aplurality of users in accordance with exemplary embodiments of thepresent invention.

FIG. 31 is an exemplary block diagram illustrating a computing devicegenerating and transmitting a first market and a second query to a firstgroup of devices in accordance with exemplary embodiments of the presentinvention.

FIG. 32 is an exemplary block diagram illustrating a computing devicereceiving a first response from a plurality of devices in accordancewith exemplary embodiments of the present invention.

FIG. 33 is an exemplary block diagram illustrating a computing devicereceiving a second response from a plurality of devices in accordancewith exemplary embodiments of the present invention.

FIG. 34 is an exemplary block diagram illustrating a computing devicegenerating and transmitting a first query and a second query to a firstgroup of users in accordance with exemplary embodiments of the presentinvention.

FIG. 35 is an exemplary block diagram illustrating a computing devicereceiving a first response from a plurality of user devices inaccordance with exemplary embodiments of the present invention.

FIG. 36 is an exemplary block diagram illustrating a computing devicereceiving a second response from a plurality of user devices inaccordance with exemplary embodiments of the present invention.

FIG. 37 is an exemplary block diagram illustrating a computing devicegenerating and transmitting a third query to a first group of users inaccordance with exemplary embodiments of the present invention.

FIG. 38 is an exemplary block diagram illustrating a computing devicereceiving a third response from a plurality of user devices inaccordance with exemplary embodiments of the present invention.

FIGS. 39A and 39B are exemplary block diagrams illustrating a computingdevice transmitting data and queries to a first group of users inaccordance with exemplary embodiments of the present invention.

FIG. 40 is an exemplary block diagram illustrating a computing devicereceiving a first response from a plurality of user devices inaccordance with exemplary embodiments of the present invention.

FIG. 41 is an exemplary block diagram illustrating a computing devicereceiving a second response from a plurality of user devices inaccordance with exemplary embodiments of the present invention.

FIG. 42 is an exemplary block diagram illustrating a computing devicegenerating and transmitting a prediction and a prediction to a pluralityof users in accordance with exemplary embodiments of the presentinvention;

FIG. 43A is an exemplary block diagram illustrating a computing devicecommunicating with a plurality of IoT devices, a third party databaseand a memory device, in accordance with embodiments of the presentinvention;

FIG. 43B is an exemplary block diagram illustrating exemplary IoTdevices in accordance with embodiments of the present invention;

FIG. 43C is an exemplary block diagram illustrating a first group ofexemplary IoT devices in accordance with embodiments of the presentinvention;

FIG. 43D is an exemplary graph illustrating an exemplary data range ofsensor values in accordance with embodiments of the present invention;

FIG. 43E is an exemplary table illustrating exemplary specificationinformation associated with an exemplary sensor device in accordancewith embodiments of the present invention;

FIGS. 44A-44G are exemplary flow charts of a process for detecting andresolving defective IoT devices in accordance with exemplary embodimentsof the present invention;

FIGS. 45A-45B are an exemplary block diagram and corresponding flowchart of a process for verifying and analyzing sensor data in accordancewith exemplary embodiments of the present invention; and

FIGS. 46A-46B are exemplary graphical user interfaces in accordance withexemplary embodiments of the present invention.

DETAILED DESCRIPTION

The present invention generally relates to a real-time eventtranscription system and method. In embodiments, the present inventionprovides a technical solution to a technical problem that currentlyexists in networks, such as crowdsourcing networks, that rely oninformation from a plurality of users connected to an electroniccomputer network to, for example, generate an accurate news report,determine accuracy of a news report, collect opinion information,predict financial market conditions, or predict stock market conditions.As discussed above, these networks suffer from a technical problem inthat they cannot identify deceitful users and prevent cheating, whichaffects accuracy and reliability of the received information.

In embodiments, the present invention is directed to a unique andnon-routine method for identifying an unreliable user(s) of a networkand/or using a user response(s) in the network to provide an accuratetimeline of an event. As further described below, at least oneembodiment of the present invention requires unconventional andnon-routine method steps that specify how to collect responses from aplurality of users, analyze those responses to determine a reliabilityrating based on the response information and create an accurate timelineof an event based on the responses while accounting for unreliableresponses.

In embodiments, a computing device (e.g., computer system) connected tothe network is configured to assign reliability ratings (reliable orunreliable or graded reliability ratings) to users based on the accuracyof the information provided in their responses, wherein a user providingaccurate information in a response is reliable and a user providinginaccurate information in a response is unreliable. Based on thesereliability ratings, in embodiments, a decision is made as to whether arespective user is unreliable. In embodiments, responses provided byreliable users are weighted more heavily in creating a timeline thanresponse from unreliable users, such that the net result is a morereliable and accurate timeline. These unique and non-routine steps offera technical solution to the technical problem posed by deceitful usersin network applications that rely on user responses.

In embodiments, as described below in Example 12 and illustrated in FIG.22 , the present invention is directed to a unique and non-routinemethod of generating an accurate news report based on informationprovided by one or more users of a plurality of users of an electroniccomputer network, the method including steps of: (a) receiving, by acomputing device, identification information associated with each userof the plurality of users of the electronic computer network; (b)storing, by the computing device in one or more databases, theidentification information; (c) generating, by the computing device, afirst stimulus message related to an event; (d) transmitting, by thecomputing device, the first stimulus message to a first group of userdevices associated with a first group of users of the plurality ofusers; (e) receiving, by the computing device from one or more userdevices of the first group of user devices, a first response, whereinthe first response includes: (i) user information specific to therespective user associated with the respective user device thattransmits the first response; (ii) responsive information related to theevent; (iii) location information associated with a location of therespective user device associated with the respective user; and (iv) atimestamp; (f) storing, by the computing device, the first response inthe one or more databases; (g) determining, by the computing device,authenticity of the first response based on one or more of theresponsive information, the location information and the timestamp; (h)assigning, by the computing device, a reliability rating to therespective user based the first response by performing steps of: (i)assigning, by the computing device, the reliability rating to be areliable rating when the location information is consistent with alocation of the event and the timestamp indicates acceptable delay; and(ii) assigning, by the computing device, the reliability rating to be anunreliable rating when the location information is inconsistent with thelocation of the event or the timestamp indicates an unacceptable delay;(i) storing, by the computing device in the one or more databases, thereliability rating; (j) identifying, by the computing device, one ormore reliable users based on the reliability rating; (k) selecting, bythe computing device, the responsive information associated with the oneor more reliable users; and (l) generating, by the computer system, anews report based on the selected responsive information.

In embodiments, as described below in Example 13 and illustrated in FIG.23 , the present invention is also directed to a unique and non-routinemethod of determining accuracy of a news report based on informationprovided by one or more users of a plurality of users of an electroniccomputer network, the method including steps of: (a) receiving, by acomputing device, identification information associated with each userof the plurality of users of the electronic computer network; (b)storing, by the computing device in one or more databases, theidentification information; (c) generating, by the computing device, afirst stimulus message related to the news report; (d) transmitting, bythe computing device, the first stimulus message to a first group ofuser devices associated with a first group of users of the plurality ofusers; (e) receiving, by the computing device from one or more userdevices of the first group of user devices, a first response, whereinthe first response includes: (i) user information specific to therespective user associated with the respective user device thattransmits the first response; (ii) responsive information related to thenews report; (iii) location information associated with a location ofthe respective user device; and (iv) a timestamp; (f) storing, by thecomputing device, the first response in the one or more databases; (g)determining, by the computing device, authenticity of the first responsebased on one or more of the responsive information, the locationinformation and the timestamp; (h) assigning, by the computing device, areliability rating to the respective user based the first response byperforming steps of: (i) assigning, by the computing device, thereliability rating to be a reliable rating when the location informationis consistent with a location associated with the news report and thetimestamp indicates acceptable delay; (ii) assigning, by the computingdevice, the reliability rating to be an unreliable rating when thelocation information is inconsistent with the location associated withthe news report or the timestamp indicates an unacceptable delay; (i)storing, by the computing device in the one or more databases thereliability rating; (j) identifying, by the computing device, one ormore reliable users based on the reliability rating; (k) selecting, bythe computing device, the responsive information associated with the oneor more reliable users; (l) determining, by the computing device, a newsreport reliability rating based on the selected responsive informationassociated with the one or more reliable users; (m) transmitting, by thecomputing device, the news report reliability rating to the plurality ofusers.

In embodiments, as described below in Example 14 and illustrated in FIG.24 , the present invention is also directed to a unique and non-routinemethod of predicting financial market conditions based on informationprovided by one or more users of a plurality of users of an electroniccomputer network, the method including steps of: (a) receiving, by acomputing device, identification information associated with each userof a plurality of users of the electronic computer network; (b) storing,by the computing device in one or more databases, the identificationinformation; (c) generating, by the computing device, a first marketquery related to past financial market conditions; (d) transmitting, bythe computing device to at least a first group of user devicesassociated with a first group of users of the plurality of users of theelectronic computer network, the first market query; (e) receiving, bythe computing device from at least a plurality of user devices of thefirst group of user devices, a first market response, the first marketresponse including: (i) user information unique to the respective userassociated with the respective user device providing the first marketresponse; (ii) past market information related to prior marketconditions; and (iii) a timestamp; (f) storing, by the computing devicein the one or more databases, the first market response of each userdevice of the plurality of user devices of the first group of userdevices from which the first market response was received; (g)generating, by the computing device, a second market query related tofuture market conditions; (h) transmitting, by the computing device, thesecond market query to the first group of user devices; (i) receiving,by the computing device from at least a plurality of user devices of thefirst group of user devices, a second market response, the second marketresponse including: (i) user information unique to the respective userassociated with the respective user device providing the second marketresponse; (ii) a prediction for the future market conditions; and (iii)a second timestamp; (j) storing, by the computing device in the one ormore databases, the second market response of each user device of theplurality of user devices from which the second market response wasreceived; (k) accessing, by the computing device, at least the firstmarket response and the second market response provided by each userdevice of the first group of user devices; (l) calculating, by thecomputing device, a market prediction related to the future marketconditions based on at least the first market response and the secondmarket rely provided by the first group of user devices.

In embodiments, as described below in Example 15 and illustrated in FIG.25A, the present invention is also directed to a unique and non-routinemethod of predicting stock market conditions based on informationprovided by one or more users of a plurality of users of an electroniccomputer network, the method including steps of: (a) receiving, by acomputing device, identification information associated with each userof a plurality of users of the electronic computer network; (b) storing,by the computing device in one or more databases, the identificationinformation; (c) generating, by the computing device, a first stockmarket query related to prior stock market conditions; (d) transmitting,by the computing device to a first group of user devices associated witha first group of users of the plurality of users of the electroniccomputer network, the first stock market query; (e) receiving, by thecomputing device from a plurality of user devices of the first group ofuser devices, a first stock market response, the first stock marketresponse including: (i) user information unique to the respective userassociated with the respective user device providing the first stockmarket response; (ii) past stock market information related to the priorstock market conditions; and (iii) a timestamp; (f) storing, by thecomputing device in the one or more databases, the first stock marketresponse of each user device of the plurality of user devices of thefirst group of user devices from which the first stock market responsewas received; (g) generating, by the computing device, a second stockmarket query related to future stock market conditions; (h)transmitting, by the computing device, the second stock market query tothe first group of user devices; (i) receiving, by the computing device,a second stock market response from at least a plurality of user devicesof the first group of user devices, the second stock market responseincluding: (i) user information unique to the respective user associatedwith the respective user device providing the second stock marketresponse; (ii) a prediction for the future stock market conditions; and(iii) a second timestamp; (h) storing, by the computing device in theone or more databases, the second stock market response; (i) accessing,by the computing device, at least the first stock market response andthe second stock market response provided by each user device of thefirst group of user devices; (j) calculating, by the computing device, astock market prediction related to the future stock market conditionsbased on at least the first stock market response and second stockmarket response provided by the first group of user devices.

In embodiments, as described below in Example 15 and illustrated in FIG.25B, the method of predicting stock market conditions may furtherinclude a step of (k) detecting, by the computing device, a tradingpattern by performing steps of: (i) generating, by the computing device,a third stock market query related to past transactions; (ii)transmitting, by the computing device, the third stock market query tothe first group of user devices; (iii) receiving, by the computingdevice, a third stock market response from at least a plurality of userdevices of the first group of user devices, the third stock marketresponse including: (A) user information unique to the respective userassociated with the respective user device providing the third stockmarket response; (B) stock ID information for a particular stock; (C)stock price information for the particular stock; (D) buy/sell datainformation for the particular stock; and (E) quantity information forthe particular stock; (iv) storing, by the computing device in the oneor more databases, the third stock market response; (v) accessing, bythe computing device, at least the first stock market response, thesecond stock market response and the third stock market responseprovided by each user device of the first group of user devices; (vi)determining, by the computing device, the trading pattern based on atleast the first stock market response, the second stock market responseand the third stock market response provided by the first group of userdevices.

In embodiments, as described below in Example 16 and illustrated in FIG.26 , the present invention is also directed to a unique and non-routinemethod of gathering opinion information provided by one or more users ofa plurality of users of an electronic computer network, the methodincluding steps of: (a) receiving, by a computing device, identificationinformation associated with each user of a plurality of users of theelectronic computer network; (b) storing, by the computing device in oneor more databases, the identification information; (c) transmitting, bythe computing device to at least a first group of user devicesassociated with a first group of users of the plurality of users of theelectronic computer network, market data, the market data including: (i)past price information; and (ii) past volume information; (d)generating, by the computing device, a first market query related to themarket data; (e) transmitting, by the computing device, the first marketquery to one or more user devices of the first group of user devices;(f) receiving, by the computing device, a first market response from atleast one user device of the first group of user devices, the firstmarket response including: (i) user identification information unique tothe respective user device and associated with the respective userassociated with the respective user device; (ii) impression informationrelated to the respective user's impression of the market data; (g)generating, by the computing device, a second market query related tofuture market parameters; (h) transmitting, by the computing device, thesecond market query to one or more user devices of the first group ofuser devices; (i) receiving, by the computing device, a second marketresponse from at least one respective user device of the first group ofuser devices, the second market response including: (i) useridentification information unique to the respective user associated withthe respective user device; (ii) prediction information related to atleast one of future price information and future volume information; (j)storing, by the computing device in the one or more databases, thesecond market response; (k) calculating, by the computing device, atleast one of a price prediction and a volume prediction based on atleast the first market response and the second market response; and (l)transmitting, by the computing device, at least one of the priceprediction and the volume prediction to the plurality of users.

Numerous embodiments provide for a client/server user query system thatallows multiple users, experiencing varying network latency, tosynchronize on a best-effort basis to absolute time events on theserver, where these may in turn have been started relative to anabsolute event that is external to the system (an example would be aseries of questions relating to the outcome of a play in NFL football,in turn synchronized to a TV broadcast). Furthermore, the server canpush content to the users in real-time, thus allowing a multitude ofclients to be presented with content in real-time on their devices(handhelds, tablets, laptops, etc. be they connected wirelessly or viawired connections) at approximately the same time despite varyingnetwork conditions.

It can be appreciated that the present invention can scale to an almostinfinite numbers of users, including for example, without limitation,human users, intelligent machine users and/or a combination of human andintelligent machine users, by employing standardized distributedprocessing methods such as those made available by various cloudcomputing services, such as, for example, without limitation, Amazon®cloud computing services and/or Google® cloud computing services.

One or more embodiments of the invention can provide generally forreal-time event transcription, cheating detection and compensation,and/or synchronizing user engagement. One or more such embodiments canbe implemented in the context of detecting cheating in an online gameassociated with one or more events. One or more embodiments can include,in the same context of an online game, some combination of analyzinglatency in individual and cohort responses, machine-trained analysis ofuser cohort behavior, and analysis of crowdsourcing information derivedfrom the same or other cohorts.

One or more embodiments can provide for one or more of: analyzingcrowdsourcing information for the further purpose of analyzing eventsthat are the subject of user responses; utilizing recursive queries toprompt secondary crowd-sourced responses; and additionally, oralternatively using intricate analysis of multiple data sources to formsynchronization across delivery paths, locales and modes.

One or more embodiments provide for a method of determining one or moreof the existence, sequence and timing of one or more event elements ofan event, including the steps of sending, by a computer devices, a firstinformation to a plurality of users in the context of an event or anevent element thereof, wherein event elements are associated with one ormore time steps and further associated with a state-transition;receiving, by a computing device, from a plurality of users one or morefirst or subsequent user inputs in reference to the event or an eventelement thereof in response to the first information, wherein the one ormore first or subsequent user inputs in response to the firstinformation is associated with a time step; sending, by a computingdevice, a second information to a plurality of users in the context ofthe event and in the context of a time step differing from the time stepassociated with the first information; receiving, by a computing device,from a plurality of users one or more user inputs in response to thesecond information; calculating, by a computing device, probabilitiesassociated with one or more of the existence, sequence and timing of oneor more event elements including the event from a plurality of userinputs; and determining, by a computing device, one or more of theexistence, sequence and timing of one or more event elements includingthe event, based on the probabilities. One or more embodiments canprovide for the first or subsequent and the second or subsequent inputsbeing derived or implicit inputs, rather than only responsive inputsthat are explicitly generated based on specific stimuli.

In various embodiments the first information sent to a set of users canbe a query that is related to an element of an event. A stimulusaffecting the user can elicit a response from the user. A query canprovide a stimulus. An element of an event that is being observed,watched and/or otherwise engaged by a user (such as, for example,interacting as a participant, as an audience member, watching on amobile device, observing from a distance, monitoring via indirect means,or engaging in any other manner, without limitation, so as to gaininformation about and form a perception of an event and/or an element ofan event) can also provide a stimulus to the user. The absolute timeand/or a receipt time relative to a reference timepoint that a usermight see a stimulus can be affected by many factors, some of which areshown in Table 1.

Table 1 below summarizes some of the factors that influence when a usersees the stimuli and how their prediction or reaction to it is affectedby delays:

TABLE 1 Inter-user Intra-user Standard Standard Factor Absolute timedeviation deviation Delay When does the Unknown High Low High userobserve the event element User Response Can be Medium Medium MediumLatency estimated Processing Can be Low Low Low Latency estimated

One of the challenges in reconstructing sequences of event elements frominputs received from a plurality of users that each have differinglatency characteristics associated with both their observation and/orengagement with an event and associated with their communications withthe event reconstruction and/or synchronization system is how to detectand compensate for the latency issues themselves and also how to detectand compensate for potential behaviors of some users who can be awarethat latency issues can provide them an advantage and may attempt toexploit their advantage for unfair individual gain or for other purposesthat the detection and compensation system and method may seek tocounter. In order to manage and overcome such challenges, one or moreembodiments provide for utilizing many types of information anddisparate sources of data related to, for example: time, location,communication mode, local conditions, content transmissions, usercharacteristics and behaviors, and any number of other types ofinformation and sources of data, without limitation. One or moreembodiments can reference data sources from a group that includes, forexample, without limitation, the following data sources:

Cellular/WiFi signal for coarse location detection;

Geo location (GPS) to determine where the user is, and possibly what theminimum transmission delay should be. This input can be used tocorrelate with other proximate users, optionally using WiFi, NFC,Bluebooth, audio content fingerprinting, or other methods, withoutlimitation, be they mechanical, acoustic, electronic or otherwise.

Mobile device accelerometer to record calculate average hand motion foruser, compare to other users and also can compare game to game for sameuser. If different, this data input can be used for flagging orindicating flagging a potentially-cheating user;

Absolute atomic time (via GPS or the mach_absolute_time( ) function (orequivalent) on mobile device;

Microphone input for audio content recognition to be used for detectionof other devices or for audio content recognition. Capture audio inputat time user confirms choice, compare to audio fingerprint of eventreference to make sure the two are within an allowable threshold, and ofother input emitted by other devices. Also compare to user-statedinformation as to broadcast media and method. If user input is at“wrong” time compared to broadcast, consider to be indication ofcheating;

“Logical” locations (venue name, social network location, or otherlocation, without limitation. This input can be used to encourage usersto “check-in” via foursquare, Facebook, twitter, etc., withoutlimitation);

Content channel (for example, ESPN, Fox Sports, etc., withoutlimitation);

Event time using the mobile device hardware clock that is notcontrollable by user, optionally by timing arrival of timing packets;

Event display time, which can be calculated according to an event timeand a user delay, such as, for example, compared to audio fingerprint ateach stimuli reaction and phone hardware clock;

Content delivery system (TV [cable, satellite, analog], radio[satellite, digital, analog], internet, mobile device [3G/4G/5G, Wi-Fi,or future wireless standards]) allows to calculate base delay dependingon range of pre-determined factors including a user's device,communication channels, and delivery method of event broadcast;

User choice (regarding an event or an event element thereof); and

User choice time (locally cached on device). For example, if user choicetime is less than a time limit stimuli response, then the system canaccept the input even if delay caused the response to be received afterthe generally allowed timeout setting.

For various purposes, including detecting and compensating for latencyand/or delay in users perceiving stimuli and detecting and compensatingfor a processing system experiencing latency and/or delay in receivinguser inputs, one or more embodiments can implement delay estimationmethods such as, for example, without limitation:

Prompt user as to the manner in which they will watch the event(radio/TV channel, can then deduce if cable, terrestrial or satellite).

Take a user's geographic location and comparing it to other users. Forexample, calculate the standard deviation of response time and/oraccuracy for users with similar location and/or provider. In, an onlinebetting game context, for instance, allow responses only within certainresponse time (for example: 2 standard deviations).

Evaluate user's response time and accuracy compared to other media (forexample: radio). If there is low correlation with a user's stated reply(for example: HDTV), but a high correlation with other media and methods(example: analog radio), then that user is more likely to be cheating.The system could then optionally employ additional algorithms to detectand compensate for suspected cheating.

At least one or more embodiments of the invention can provide for thefollowing methods or sub-methods for sending query information to a setof users and receiving responses from that or another set of users or acombination thereof:

Randomization: ask the questions in different order, and place theresponses in different areas of the screen.

Honeypot testing for other transmission modes: Ask questions that theuser would not be able to legitimately know the answer to (for example,if user claims he's on a satellite feed, pop up a question about acommercial that showed up on a cable feed—if user responds correctly tomany such honeypots, it means they are not observing with thetransmission media and method(s) they claim and with the delayassociated with that transmission media and method(s)).

Control cohorts (such as, without limitation, in cheating context),e.g.: (a) Control group of known users that don't cheat; and/or (b)Control group of known users that actively cheat.

Calculate average response time based on an aggregation of mean timesfor all of the users' responsive actions to various stimuli (forexample, see the event on screen, think of a response, send a response,and receive the response in the system. This will generate a certaintime range, with a minimal “floor” time. If the system receives ananswer before this time, it is probable that user is attempting to cheator defraud.

Calculate average correct response percentage. For example, if user isfurther than two standard deviations of that percentage (p-factor of0.05), then user is probably cheating.

Take user's geo location and compare it to other users. Calculatestandard deviation of response time and/or accuracy for users withsimilar location and/or provider. Reactions to stimuli are only acceptedonly within certain response time (for example: 2 standard deviations).

Take user's response time and accuracy compared to other media (forexample: radio). If there is low correlation with user's stated reply(for example: HDTV), but high correlation with other media and methods(for example: analog radio), then the user is likely to be cheating;and/or

Utilizing accuracy measure(s) based on probabilities and successes,e.g., A(i)=average[p(t)*Si(k, t)].

At least one embodiment of the invention can provide for a system thatcan send and/or display different questions (stimuli) to differentgroups of users in the context of an event or event elements (timing anddistribution of questions determined by the system) and then correlatethe responses from the users (both time of indication and answer), inorder to predict with a high likelihood what happened and when in thecontext of the event and event elements. The system can use,recursively, one or more sets of responses to generate the choicesdisplayed for additional stimuli (and the system can combine informationfrom other data sources with the one or more sets of responses in acombined analysis for this purpose). Additionally, the system can adjustthe types and/or instances of system-distributed stimuli for sending todifferently constituted set or group of users as selected by the systemat any specific time based on the system processing data inputs. Forexample, without limitation, a question may be any one of the following,each sent to a system-selected group of users:

-   -   “Will John Doe wear a jacket when he next appears on stage?” to        which the displayed list of possible answers may be “Yes” or        “No.”    -   “What color dress did Jane Doe wear when she appeared on stage,”        to which the displayed list of possible answers may be “Green”        or “Other.”    -   Upon selection by a large group of users that Jane Doe wore an        “Other” colored dress at any particular time of the event, it is        therefore likely that she appeared on stage at such a point.    -   Another case, for example, without limitation, in the context        American NFL Football, can display one or more of the following        questions to a plurality of users (timing and distribution of        questions determined by the system):    -   “Will the next play be a rush or a pass” to which the displayed        list of possible answers may be “Rush” or “Pass.”    -   “How many yards were gained?” to which the displayed list of        possible answers may be “more than 10” or “less than 10”    -   “How many yards were gained?” to which the displayed list of        possible answers may be “more than 5” or “less than 5”    -   “How many yards were gained?” to which the displayed list of        possible answers may be “less than 5” or “fumble”

Correlating this data over a large body of users can preferably providevery high and even more preferably provide almost absolute precision tobe attained, especially when the system employs any combination of anyvariety of algorithms (including, without limitation, heuristics, combfilters, auto-correlation, statistical algorithms, machine learningalgorithms, multi-agent systems, and so on) used to improve theprecision of the derived data. The processing method according to one ormore embodiments can also feed the various stimuli of similar events(past of present) as additional signals into the algorithm. As well, thesystem can employ a variety of algorithms to determine which users shallbe shown which questions at whatever time.

Crowd-Sourcing

One or more embodiments can provide a system for using crowd-sourcing,wherein the system can determine which event occurred according to userreplies (based on a plurality of replies from a plurality of distinctusers) and can then dynamically update the decision model. Based on thetime the user replies, the system can estimate the user's location,transmission method and media (TV, radio, live event, etc., withoutlimitation) and other parameters.

The system can compare the users' replies to those replies of otherusers with, in one or more embodiments, similar characteristics, forexample: affinity (including team and player preference, brandpreference, stock market trading strategies, financial productpreference, industry sector focus, etc.), mobile device metadata(including connection method, broadcast viewing method and so on),purchasing history and geographical location among others (suchcomparisons determined by the system in accordance with an embodiment),and the system can also use a checking process if the system determinesthat the user is likely to be cheating (for example, if user says he orshe is watching an event on TV, yet the behavior and response time ofthis user is similar to a typical profile of a user listening on aradio).

An embodiment can provide for the system to produce a transcript of theevent, accurate to a very small time interval, that outlines thesequence and timing of events that happened by using either onsite orremote viewers that reacted to various stimuli during said event, andthen algorithmically deriving the sequence and timing to produce thetranscript with minimal delay. Crowdsourcing techniques may be appliedin such computations. For example, output from the system can be used toverify accuracy of other stated transcripts (such as those provided bythird parties). Furthermore, output from the system can be further usedto verify and mitigate cheating attempts by users. Additionally, outputfrom the system can be used to provide an accurate and timely data-feedof the event sequence which can be transmitted to other third parties).

One or more embodiments provide for a method and a system by which theabsolute time that a stimuli (or event element) occurred can beaccurately deduced from a plurality of data without any physical nexusat the location of the occurrence of said stimuli. For example, remotesensing and image processing can add to a data stream and provide timinginformation. As a further example, without limitation, one embodimentcan use machine vision to determine when a snap happened in a footballgame. This determination can allow for increased accuracy when computinglatencies because the processing has an external data source that cannotbe controlled nor affected by users. See Machine Vision(http://www.sciencedirect.com/science/article/pii/S0262885609002704) andaudio monitoring (to detect waveforms that indicate the event happenedand, optionally allow timing synchronization) and Acousticfingerprinting (See Duong, N. Q. K.; Howson, C.; Legallais, Y, “Fastsecond screen TV synchronization combining audio fingerprint techniqueand generalized cross correlation,” Consumer Electronics-Berlin(ICCE-Berlin), 2012 IEEE International Conference on, vol., no., pp.241,244, 3-5 Sep. 2012; doi: 10.1109/ICCE-Berlin.2012.6336458), whichforegoing references are herein incorporated by reference in theirentirety.

One or more embodiments provide for receiving crowd-sourced inputs fromremote viewers, wherein a plurality of remote viewers, each viewing theevent over any transmission method and media with any propagation ortransmission delay endemic to such transmission method and media, canenter their inputs as to the exact and/or absolute time that theywitnessed the stimuli occurring (the “epoch”). Any number of suchindications, from any number of users, can then be calculated in aplurality of ways, including, for example, by designating certain usersas trusted users who view the events and indicate when stimuli occur,with said such trusted users viewing the event over a transmissionmethod and media with a known and predicable propagation or transmissiondelay. With such a plurality of trusted users giving a plurality ofindications, the present invention can be used to calculate when thestimuli occurred by subtracting the known transmission or propagationdelay from the absolute time indicated by said power users and by adjustfor the human reaction speed of the users both individually and as agroup.

One or more embodiments provide for receiving crowd-sourced inputs fromlocal viewers. Similar to crowdsourcing from remote viewers, a pluralityof trusted users who are known to be in the immediate proximity to thestimuli can indicate when the stimuli occurred, without any transmissionor propagation delay. In this case, only the adjustment for the humanreaction speed of the users both individually and as a group need betaken into account. Additionally, the system can utilize thegeo-location functionality built into contemporary smartphones (usingGPS and other methods) to determine these users are physically proximateto the actual event and thus have minimal latency. The system can thensend timing packets (for example as provided by the ping utility and theICMP protocol, http://tools.ietf.org/html/rfc4884 and its predecessorsand citations, herein incorporated in their entirety) to determine thedata transmission latency between a user's device and one or moreservers executing computer code of an embodiment of the presentinvention.

Synchronizing Input

One or more embodiments provide for synchronizing input from a largenumber of devices to ascertain content, time, and time differences fromvarying external stimuli. In one embodiment a viewer can see an event attime, t(e) (or t.sub.event), a viewer can respond to an event at a timet(r) (or t.sub.response), and the system and/or method can process theviewers' responses at time t(p) (or t.sub.process). In embodiments, aplurality of data sources (such as, for example, without limitation,wireless signal, GPS, device accelerometer, absolute time, microphoneinput, logical location, delivery system base delay, event time,calculated event time, user event-based choice, and user choice time)may be made part of the processing. In one or more embodiments, a delayestimation can be made by: (i) user-stated mode, (ii) geo-location,(iii) ping times and/or (vi) comparing response time and accuracy toother media/modes. The system and methods can determine, optionally inreference to a standard and precise clock time, calculate and/orgenerate latency analysis based on t(r), on t(e), or on a differencebetween t(r) and t(e). Various embodiments can further provide foradditional methods of accounting for various latency between usersseeing the questions sent to them by an embodiment of the presentinvention, and [the embodiment] receiving the user responses to theaforementioned questions. The system and method of numerous embodimentsprovide for measuring (i) how long before a user sees a question appearand (ii) how long it takes a user to respond to the system based onsystem-prompted and/or system-registered stimuli. This, together withother latency data the system and/or method can acquire or derive,allows for more accurate latency compensation and cheating mitigation.One or more embodiments can further adapt the weights governing theprobability of a certain plurality of users responding to variousstimuli, using various inputs and parameters as inputs for recalculatingthe weights at any point.

One or more embodiments can provide for identifying control groups withbetter latency, such as those users who are physically present at theevent location and thus are not subject to propagation delays of thebroadcast transmission system. Further, users using mobile devices thatallow for lower latency transmission can also be identified. Lowerlatency is desired because higher levels of statistical accuracy can beattained within a shorter time frame, enabling the filtration ofincorrect/fraudulent responses faster and more accurately. In one ormore embodiments, less latency is better. Less latency also allows forfaster training of the model and lower risk of cheating, and it confersthe added benefit of using the lowest latency and the minimum timemarker for which an event happened. This is especially important if thisrapid training happened as a stimuli response by users (one or more) whocan be determined (by location) to be physically proximate to the event.

Detecting and Synchronizing Input Timing

One or more embodiments provide for detecting and synchronizing inputtiming with neighboring devices by a variety of methods, including,without limitation, WiFi, Bluetooth®, NFC, and/or similar methods. Anembodiment can have each device emit specific sound (within and/oroutside the audio band) detected by other devices' available sensors.Another or the same embodiment can use both input timing, as describedabove, and audio content recognition (such as that provided by Duong etal., as previously cited herein)) to determine which method and media isbeing used to broadcast the event to the viewers, and to further aid inthe detection of cheating attempts. An embodiment can use input timingto identify control groups with better latency.

Advanced Processing Methods

One or more embodiments can provide for an ability to use a variety ofadditional processing methods in the transformation of, for example,without limitation, inputs, stored information, analyzed and calculatedinformation and/or derived or generated information, including machinelearning, heuristics, pattern-matching, game-theory, and otheranalytical and/or processing methods.

Determining Absolute Time

One or more embodiment scan further provide for determining absolutetime (and nature) of stimuli without physical presence at location(s) ofstimuli. This can further include, without limitation, one or more of:displaying different stimuli to different groups, then correlateresponses to predict what happened where and when; recursively using ofone set of responses to generate additional query/stimuli; processingbeing automated within the system; and using heuristics, comb filters,auto-correlation, statistical algorithms, machine learning algorithms,and/or multi-agent systems, inter alia, without limitation, in order toimprove the precision of the derived data. The processing in at leastone embodiment can one or more of: feed other stimuli of similar eventsas additional signals (past or present); dynamically change decisiongraph (represents options for events that have taken and/or are takingplace (and potential future events)), wherein crowd-sourcing responsescan be used to prune or exclude decision-tree branches; compareindividual replies to group to detect cheating or fraud; produce one ormore transcripts of one or more events; and/or identify one or morecontrol groups with better accuracy and/or latency.

EXAMPLES

The following examples illustrate embodiments of the present invention.The following examples are not intended to be limiting. It will beappreciated by those of skill in the art that embodiments may be appliedto other use cases not specifically called out herein without departingfrom the present invention.

Example 1. Dynamically Generating (Promotional and Other) Content Basedon User Feedback

Assume that a major sporting event currently takes place. Millions ofpeople are watching it on their television sets and online. By queryinga sufficiently large number of people (and by filtering the fraudulentones using the method described herein) the system and methods of a oneor more embodiments may be able to determine the following for veryshort time intervals:

How far was a basketball player from the basket when he shot?;

How many yards were just gained in a football game?;

Which team seemed favorite at any given moment?;

What was the color of the dress worn by a leading celebrity when shewalked on the red carpet?; and/or

Exact play by play outcomes of each play within the event.

Example 2. Reconstructing News Events and Providing Insights

Using large amounts of user responses to stimuli in short intervalsthroughout the course of an event, (including press conference,disaster, or anything being reported upon) the system and methods of oneor more embodiments are able to faithfully recreate the event. Inaddition, the system and processing steps are able to do the following:

Identify points of specific interest (based on user feedback) andhighlight them

Generate automatic summaries containing details such as highlights, MVPsand common perceptions

Provide recommendations to journalists and other news services regardingthe best way to portray an event in order to “cater” to the perceptionsof their readers.

Provide accurate timeline of the event with high accuracy and precisionand low latency.

Example 3. Ascertaining the Sentiment of an Individual User

One or more embodiments provide for using various sources of informationand applying sub-methods described above for ascertaining the sentimentof an individual user. The system enables to accurately predict theusers' affinity to various teams, actresses, etc. and withoutlimitation, based on the users' selections, group assignment, or otherdata, inter alia. Further, by evaluating what may happen in theimmediate future, the system in one or more embodiments can predict ifthat specific user will be happy or sad in the next few seconds. Thisprediction has extremely high value as an input to any number ofadvertising and marketing systems and campaigns.

Example 4. Users Creating Stimuli Questions and Response Options

One or more embodiments may allow users to create stimuli questions andpossible response options themselves, optionally in real time. Inembodiments, the system may then query a plurality of users with thecreated questions, optionally in addition to the generated questions.One or more embodiments may enable a first user to communicate a seconduser or with a plurality of other users via written, audio, video, orother means.

Example 5. Event-Reporting Channel

An additional embodiment can provide for an event-reporting channel,whereby the system and/or methods leverage a user-confidence trackingfeature. Such an embodiment can allow a cohort of reporters, eachreporter associated with a continual evaluation of each reporter'sindividual confidence levels, to be used to crowd-source input from aplurality of sources. These responses can be processed using the methodsdescribed herein to capture various points of view and evolution ofsentiment and information over the course of the reported period. Theseevents can also be processed by methods provided for in one or moreembodiments, which methods produce an accurate transcript of timeline ofthe reported event, by correlating the occurrences with the highestlikelihood that were deemed to have occurred and had been reported on byusers of the highest confidence and lowest latency.

Example 6. Financial Trading

Financial markets constantly publish a wide assortment of data aboutmarket status and various indicators, including future and past analysisof indicators, trends and events. The high volume of data, the rapidexpiration of old data and the absolute volume of data produced andreport make it extremely difficult to track in an actionable manner. Oneor more embodiments of the present invention can provide for querying aplurality of users as to each of their impressions about released data(past impressions), and to further query a plurality of users as to whatthey think will happen in the future (future predictions). The systemcan process such user responses, optionally together with external datasources to enable accurate prediction of, for example, future financialinstrument or commodity prices, without limitation, and to optionallyeither signal trades accordingly or to sell access to the predictions,again without limitation.

Example 7. Stock Exchanges

As with financial markets, stock exchanges also publish a wideassortment of data about market status and various indicators, includingfuture and past analysis of indicators, trends and events, includingstock prices, periodic financial reports from companies, optionconversion dates and other company information, without limitation. Thehigh volume of data, the rapid expiration of old data and the absolutevolume of data produced and report make it extremely difficult to trackin an actionable manner. Furthermore, users may be further influenced byreading company reports, leaks, news media, social media, etc., all ofwhich may have an effect on the price of any traded stock. One or moreembodiments of the present invention can provide for querying, bycomputing device, a plurality of users as to what is each of theirimpressions of released data (past impressions), and to further query,by computing device, a plurality of users as to what they think willhappen in the future (future predictions). The system according to suchan embodiment provides further for processing such user responses, by acomputing device, optionally together with external data sources, andgenerating accurate prediction of future economic and/or market eventsand/or event elements, such as, for example, without limitation, futureprices of financial instruments or commodities, future likely changes tothe price of company shares, future trading volumes, and future shortand long positions. Further embodiments can provide for signaling tradesaccordingly and/or selling access to the predictions, again withoutlimitation.

Example 8. Market Research

A market research or similar company can use large amount of datacollected by an embodiment of the present invention to acquire insightsas to customer sentiment towards various products, current events,and/or other trends. Such company may entice users to use variousapplications, including such applications that perform as client devicesfor the present invention, or as various computer games in order to sendqueries to users and analyze their responses.

Example 9. Advertising

A promotion or advertising company can use one or more embodiments ofthe present invention to send advertising to users while they interactwith various query and response cycles as provided for by the presentinvention. Such users, using one or more embodiments, may interact witheach other, with celebrities, or with other users as described inEXAMPLE 4, without limitation. The promotion or advertising company maychoose to incentivize any user to further interact with the system.

One or more embodiments can be understood in more detail and with moreparticularity by reference to the further processing approaches andsteps that follow below and with illustrative reference to the Figures.

In at least one embodiment, the processing can be illustrated by lettingE represent an entities graph, and letting e.1, e.2, . . . e.n be entitytypes 1 through n. Similarly, let e.1.1, e.1.2, . . . e.1.m be a set ofm entities of type e.1. And let entities graph E be infinitelydimensioned.

FIG. 1 illustrates an initial set of connections between a plurality ofentity types according to one or more embodiments of the invention.Three different entity types are illustrated, 101-103, each includingsome exemplary member entities. Connections 104 and 105 depict theinitially-available connection between the illustrated entities and eachof the entity types. Similar graphs can be created for each type ofevent (e.g. football game, Academy Awards ceremony, without limitation)for which one or more embodiments of the present invention would apply.For each such event, a generic ontology can be created for definingconnections between entity types which may happen at each occurrencewithin an event.

For example, still referring to FIG. 1 , without limitation, for theevent “Academy Awards Ceremony”, entity of type “actress” 101, withentity type “actress” 101 denoting which entities of type “actress”(Angelina Jolie being actress.1, Glenn Close actress.2, and so on); with102 denoting “attire” as the entity type such that “Blouse” is denotedas attire.3, and so on). Similarly, 103 denotes various colors, suchthat “Green” would be color 3. Entity type “actress” 101 can beconnected to entity type “attire” 102 by connection 104 of type “iswearing.” Entity type “attire” 102 can be connected to entity type“color” 103 via connection 105. The initial connections are {n:n}, whichmeans that if a connection is defined between entity types a and b, allthe entities in the former group are initially connected to all entitiesin the latter.

It should be understood that a connection between entities can, in oneor more embodiments, be considered as a relationship descriptor orpredicate (arc) that relates two objects (or nodes) in a semanticstatement, or a semantic triple-store, such that numerous additionalmethods of database storage, including graph-type databases, can beused, and which can allow very rapid analysis of the graph-storeutilizing multi-threading processor architectures, optionally arrangedin any distributed processing topologies.

In embodiments, during the training phase of the model (or by analyzingprior events, and/or by applying defined rules for the event type, forexample the official rules for a football game), for each point in time(which can alternatively be referred to as an “occurrence” or as an“event element”), the system and/or method can use the predictions of aset of users as to what will happen in order to assign weights to thevarious connection between the entities (the strength of the outgoinglinks from each entity are normalized to 1). The weights can be arrangedas templates so that no prior knowledge about the particular actors ofan event need be required; all that is needed is a general knowledge asto the type of the event.

Still with reference to FIG. 1 , when creating entity types 101, 102 and103, for example, and defining connections 104, all entities areinitially connected. Based on the predictions of a set of users, onlysome of the links will be retained (e.g., all connections havingstrength below a predefined threshold or zero can be discarded). Onceall valid connections have been defined, one or more embodiments maycalculate the probability of each combination of connections to occur bymultiplying the probabilities of all of its “elements” happeningtogether (such as, for example, the probability that actress AngelinaJolie will arrive at the Academy Awards, the probability that she willarrive wearing a dress, and the probability that the dress will beblue).

The training process can be included of two distinct processing aspects.The system and/or method of one or more embodiments calculate (1) theusers' predictions about future occurrences as well as (2) theirimpressions about past occurrences (the very near past, such as, forexample, without limitation, sometimes only seconds in the past). At anygiven time, t, users can be presented with two series of questions: (a)questions that ask them to predict what will happen next (i.e., forexample, what will be the next occurrence(s) at time t+1, t+2, or atlater time points); and (b) what was the occurrence that took place attime t−1, t−2, or earlier time points. The latter question set alsoconsists, in addition to “legitimate” questions designed to infer thesequence of occurrences within said event, of questions designed todetect frauds and intentionally incorrect answers.

Although information about past occurrences and events is usually moresolid, the predictions about future occurrences offer at least twoimportant advantages: (1) they encourage users to participate, as a sortof competition (which one or more embodiments may further realize as agame application of any kind); and (2) they confer additionalinformation (which is likely to be more abundant, because of theadditional interest) that could be augmented into the data analysis.

By using a “sliding window” technique, the state of each occurrencewithin an event at each time t is updated at least twice—its initialstate is set in time t−1 (as a prediction about the future) and is thenmodified again in time t+1 (as a question about the past). It should befurther understood that either t+1 and/or t−1 can be t+p or t−q,designating any point in the past or the future). In addition, eachstate may be updated n additional times with n being the size of thesliding window. In embodiments, a Hidden Markov Model (HMM) algorithmmay be used to update and correct the states sequence. Alternatively,other embodiments may use other kinds of algorithms. In embodiments,even though a lower weight may be allocated to predictions about futureoccurrences in comparison to impressions about past occurrences thatalready happened in the calculation of the probabilities and states,they nonetheless, in embodiments, play an important role.

It is important to note that multiple stimuli questions relating to thesame occurrence may be created. For example, an occurrence of “KobeBryant shot a three pointer from 25 feet” may be a part of an almostidentical set of stimuli questions, each specifying a slightly differentdistance. Each element in the question may vary, which of course leadsto a large number of possible states. The likelihood of each state canbe managed and determined by the algorithm or algorithms used in one ormore embodiments, including without limitation Hidden Markov Models,Conditional Random Fields, and other suitable methods It should befurther noted that multiple algorithms may be used in parallel to createa multi agent system, optionally also using a “mixture of experts” orother ensemble methods to obtain better performance and/or accuracy. Itshould be further appreciated that the processing of any step or moduleof the present invention need not be limited to a single applicationrunning on a single computing device, and that numerous parallelprocessing and scalability can be applied to embodiments of the presentinvention.

During the training phase, in embodiments, the process may compare thepredictions (stimuli responses) made by a set of users to theoccurrences that actually transpired in the real world and match theirprediction responses to “known good” input sources, such as humantaggers and other sources of information that become available after anoccurrence has been actually happened. A “human tagger” is a person orplurality of people who refine the data captured and processed by thesystem using human intelligence as well as their personal knowledge andexperience as it applies to the analyzed event and its entities, all inorder to enhance the accurate of the data during the training phase.When the system is operating in runtime phase, the process alreadypossesses a large states matrix and information about variouspredictions. For this reason, each iteration only needs to calculate thevarious probabilities of each observation and it can “predict the past”almost instantaneously after it has occurred. In addition, by analyzingpast occurrences, the process can calculate the “real” probabilities (orat least close enough approximation) of options that are likely to occurduring an event.

In embodiments, a system that has an elaborate decision graph or statetransition diagram, dynamically changing as a result of a live event, isprovided. The event can further include multiple occurrences or eventelements. Such a decision graph can represent options for events and/orevent elements that have taken place and/or are taking place, as well asevents that have not yet happened, but which may occur in the future.Some branches of such a decision graph according to one or moreembodiments can be mutually exclusive options; i.e., if one optionoccurs, then the other option(s) certainly do not occur. Therefore, if aderivative of such an option occurs, then the system can identify withvery high probability that the other branch in the decision graph, andall of its derivatives, have been eliminated. For example, if it isgiven that A and B are mutually exclusive decisions (branches) on thesame decision graph, with AA and BB being derivative decisions ofoptions A and B, respectively, then if the system identifies that eventBB occurred, the system can calculate with high probability that theentire A branch, and all subsequent options (including AA), have beeneliminated. This can be further illustrated by an example, as follows:The system is asked to determine whether the weather today is sunny orcloudy. However, if during another question the user replies that it israining, then certainly the system can identify that it is cloudy today.

Referring now to FIG. 2 , a decision graph 201 depicts a partialdescription (or an illustrative subset) of the different states andtransitions that may occur during a game of NFL Football. It can beappreciated by a person skilled in the art that such a graph may bedesigned using a plurality of applications, such as, for example,without limitation, Bonitasoft® brand (version 5.5) business processmanagement (BPM) or workflow software, and that the such graphs,optionally implemented as workflows, may be programmatically operatedupon by applications such as, for example, without limitation, ApacheSoftware Foundation's jBPM (version 5.4) software package.

One or more embodiments can provide for modelling the problem describedabove as a Hidden Markov Model (HMM) problem, where a states transitionmatrix can be created by transcribing the rules, optionally using adecision graph or state transition diagram, of the event in questioninto computer-operable instructions. Still referring to FIG. 2 , a statemapping or decision graph 201 can represent multiple elements of anevent that can flow sequentially from another event element, and/orwherein legal transitions exist from one event element to another.“Legal transitions,” as used here signifies those transitions in statespace allowed by the rules of a game and/or allowed by the known and/orcalculable physical, transactional, logical and/or other constrainingrequirements associated with a set of one or more event elements thatinclude an event state space. The decision graph 201 can be optionallyrendered as a state transition matrix, that can be further generated bya training phase (analysis of past events to determine legaloccurrences, the likelihood of each occurrence, and optionally furtherrefined by “known good” information such as a transcript of a priorevent, comparison to a television recording and/or other data feed). Inone or more embodiments, the states transition matrix and theobservations matrix (and their probabilities to be connected with eachstate) can be further generated by analyzing a set of users'predictions. Further information about Hidden Markov Models (HMM) can befound at http://www.cs.sjsu.edu/faculty/stamp/RUA/HMM.pdf, hereinincorporated by reference.http://cran.r-project.org/web/packages/HMM/index.html.

HMM's can be further understood and implemented by one of ordinary skillin the art by reference to Ghahramani, Zoubin; Jordan, Michael I.(1997). “Factorial Hidden Markov Models”. Machine Learning 29 (2/3):245-273. doi:10.1023/A:1007425814087 (which is hereby incorporated byreference in its entirety).

According to one or more embodiments, the system can use HMM todetermine the most likely set of events based on the set of observationsprovided by users over a certain span of time. By analyzing theobservations and the probabilities of transitioning from one state toanother (that is, for selecting which occurrence will happen next basedon what has actually happened during an event), the system can determinethe most likely set of states to have occurred and also correct itspredictions when new information becomes available.

Sampling Intervals

in embodiments, the size of the interval that will be used for samplingwill be determined based on one or more of the following factors: (1)Number of available users: Preferably, the number of users available tothe system in one or more embodiments is in the range of 10 to 50million, more preferably the number of users available to the system isin the range of 10 to 100 million, and most preferably the number ofusers available to the system is in the range of 10 to 50 billion.However, it is understood that some embodiments of the system and methodcan allow for as few as any plurality of users. (2) The number ofpossible likely outcomes—based on the states transition matrix andpreviously collected observations the system can determine when therange of possible future states is small or large. A small set ofpossible outcomes will enable an embodiment to approach a smaller numberof users and query them with a larger interval. Preferably, the numberof likely outcomes is in the range of 1 outcome to 1 million outcomes,more preferably the number of likely outcomes is in the range of 1outcome to 10 million outcomes, and most preferably the likely outcomesis in the range of 1 outcome to 1 billion outcomes. It will beunderstood that the system and method of one or more embodiments allowfor a number of likely outcomes in the range of 1 to more than 100billion outcomes. (3) How dynamic is the event—events that are moredynamic, for example basketball or football games require more frequentsampling than a chess tournament. As such, a higher sampling frequencywill be required to achieve sufficient accuracy, similar toNyquist—Shannon sampling theorem (see Jerri, A. J., “The Shannonsampling theorem—Its various extensions and applications: A tutorialreview,” Proceedings of the IEEE, vol. 65, no. 11, pp. 1565,1596,November 1977 doi: 10.1109/PROC.1977.10771; incorporated by referenceherein in its entirety). A larger number of human users is also desirousin such cases, as it would be difficult for a small number of humanusers to respond sufficiently quickly and accurately to maintainstatistical accuracy by the system. Automated users may of courserespond faster. The more dynamic the event, the more users need torespond in order to get good accuracy—in chess, for example, one usermay report and the system may have 2 minutes to do it per move, with avery limited number of options (based on the state of the chess board):so perhaps in chess an approximate 5 legal moves and 2 minutes areneeded to decide. By contrast, in the final minute of a basketball gamethe method prefers to have a lot of users each responding to a verysimple question, and the method prefers a lot of various questionsbecause the numbers of options is very large and the rate of change isalso very large. Preferably, the frequency in which the system can sendquestions and receive responses to/from users is in the range of 5 to 20seconds, more preferably the frequency in which the system can sendquestions and receive responses to/from users is in the range of 0.1seconds to 10 minutes, and most preferably the frequency in which thesystem can send questions and receive responses to/from users is in therange of 1 picosecond to 100 days. (4) The attributes of availableusers—the users' location, level of reliability, among other attributes,can all be determining factors in the decision on sampling interval.

Sampling is performed by segmenting the users into groups and thenquerying the groups of users, by assigning a single query to each groupper cycle, and with the understanding the multiple cycles can beexecuted in parallel, either to the same set of groups or to any set ofgroups. Each group will be assigned a single “observation” per suchcycle and asked to respond about its occurrence. Groups may be queriedfor more than one type of observation at a time. The questions sent tousers may be direct or indirect ones; indirect ones being used to maskthe actual question, but the users' answers to them enable, at minimum,deduction of a “yes” or “no” answer to the predicate.

Table 2 below illustrates an analysis of the responses of the users ineach group and the calculation of the probabilities.

TABLE 2 Yes No Group Observation Answers Answers Probability Group_1Player X shot from 900 100 0.9 15 feet Group_2 Player X shot from 600400 0.6 25 feet Group_3 Player Y shot from 300 700 0.3 15 feet

It should be understood that Hidden Markov Models are but one way toimplement these steps, and that other algorithms may be available or maybecome available in the future, all of which are to be considered withinthe scope of the present invention.

Referring now to FIG. 3 , a generic process of using Hidden MarkovModels is depicted, enabling the process to revise its predictions withthe arrival of new information. This is especially useful in the case ofsports games where the results of a play may be changed due to ajudgment call by the referees, or in any other type of event when aseries of event elements has one or more elements that can be altered(by after-the-fact revision or by updated calculations with more data).At least one embodiment provides for analyzing event element group 301,wherein the process, analyzes user predictions of a successive number ofevent element occurrences (shown as an occurrence of “1” transitioningto an occurrence of “2,” then transitioning to another occurrence of“1”) in order to estimate the probability of the next event elementoccurrence 304. In event element group 302 the process has used theobservations and the transition matrices in order to determine the mostlikely state of the next event element, 305, which in this illustrationis an occurrence of “4.” Then, by analyzing the process as a whole inevent element group 303, the process can determine that the predictedstate of the third event element (or node) 306 of the actual eventsequence needs to be updated, in this illustration updating anoccurrence of a “1” to and occurrence of a “3.”

(1) User Credibility and (2) Time Intervals

At least three major elements dictate a set of users questioningstrategy: one element first is how dynamic is the event in question: forexample, a football game is much more fast-changing than a chesstournament. A second element is the time that has passed since the userhas been presented with the question, and a third element is the user'scredibility. While the first element is quite straightforward, the othertwo elements can be further explained as follows:

(1) The Time that has Passed Since the Event

A prediction about the future that has arrived long after the eventitself is of no use to the system. A simple method, such as provided forby one embodiment, could define a simple rule stating that allpredictions that arrive after the event are ignored. However, thisapproach will not be taking other elements into account: delays inbroadcast times are the simplest example, along with geographicdistance, among others. Communication delays are also a legitimatereason for a delay in the response.

In embodiments, an array of features designed to deduce whether or notthe prediction is “legitimate,” may be employed. The array of featuresmay include, for example, one or more of the GPS location of the phone,the timestamp of the message, the medium used by the user to view theevent and the user's credibility score, without limitation.

In embodiments, one or more algorithms may be utilized and/or executedto perform the functions responsible for determining whether or not toprocess an answer returned by the user are the Wait_For_Response( )function, which records the time the response was captured or returnedand the Calculate_User_Response_Time_Fraud_Likelihood( ) function, whichreturns a number that reflects how legitimate the user's response timeis, which number is a credibility metric that is then then used this totrain the model, group cohorts, and otherwise dynamically adjust thealgorithm and which also takes into account how dynamic is the event andthe user's credibility.

(2) The User's Credibility

The credibility of the user is determined by several factors: (a) Pastbehavior—by comparing the user's responses to the actual events (eitherevents a set of human taggers determined happened or events which had ahigh degree of certainty), the method can obtain a solid estimator ofits reliability; (b) Meta-data features—if the user's metadatacontradicts the claims made by that user (for example, the user claimsto have seen the event “live” while he was 100 km away in this casewhatever score he had would be multiplied by zero—he's a fraud; and (c)Trapped by “honeypot questions”—part of the query mechanism generates“honeypot questions”, questions that are known to be false and whoseonly use is to identify users who are not being true. For example, aquestion may refer to a commercial not shown is the user's claimedlocation or refer to a weather condition that did not occur.

One or more of the aforementioned factors, without limitation, can beused to determine the user's “credibility score,” along with other dataoperated in embodiments of the present invention. This score will factorthe percentage of truthful answers to honeypot questions, the averagelikelihood (assigned by the HMM) of the observations made by the userand how dynamic was the event in question (dynamic events are harder tocall correctly). For example, comparing the last minute of an NBAbasketball game versus the last move of a chess match: In basketballthere could be (for example) 17 different options that occur in the next(for example) 10 seconds, whereas in chess there could be (for example)3 options in the next (for example) 2 minutes.

As such, low latency and high accuracy are very important for the formerbasketball example, in order to provide accurate statistics andmeaningful probability assessment in a minimum amount of time. Having alarge body of users will generally decrease average latency, as someusers will respond more quickly than others, therefore enabling thesystem to use a user credibility assessment metric in order to qualifythose users that respond quickly and accurately.

A person of ordinary skill in the art can appreciate that the proposedframework for the calculation of the credibility score can be NaïveBayes or any other classifier, which is capable of taking advantage ofthe probabilistic nature of the HMM and produce a single value (rangingfrom 0 to 1) representing the user's credibility. One can furtherappreciate that in order to simplify the examples herein, a credibleuser is arbitrarily defined as one that has achieved 80% accuracy in thelast 30 minutes, with any further indications of possible fraud. It isfurther appreciated that in embodiments of the present invention mayimplement the credibility assessment function illustratively representedin the algorithm by the Calculate_User_Response_Time_Fraud_Likelihood( )function, which factors a plurality of elements in order to determine ifa user is suspected of fraud and to optionally decide whether or not touse the responses provided by the user. For example, a first user mightanswer 90% correctly over an interval of 30 minutes, however theirlatency might be more than one standard deviation away from the averagelatency for the group that first user is assigned to, and the firstuser's physical location has been determined to be outside of the eventvenue, such that the process can generate a decision that the first useris viewing the event via some broadcast medium with at least thatmedium's inherent latency and/or propagation delay. In contrast, asecond user who answered only 80% accurately but has been determined tobe inside the event venue and the second user's latency was less thanhalf of one standard deviation away from the minimum latency of any userin the system is assumed to be reliable enough and accurate enough forthis example purpose.

One or more embodiments of the invention can provide for a series ofsteps of processing information that are accomplished by machineinstructions directing a computer processor. The software program of onesuch embodiment can be further described below in Tables 16-26, whereinthe first column lists the line number (and matching figure and step) ofthe algorithm, the central column includes descriptive commentary on thefunction of the software step and the right column depicts pseudo codeas may be realized by at least on embodiment of the present invention.

The system can leverage what it knows about what happened or is about tohappen and combine that with its analysis of the user's responses toknow what teams, actors, and so on (without limitation) the user favors.Then, the system can leverage the propagation delay of the user toanticipate if that user will be happy or sad in the near future and sendthat signal to an advertising system that will offer that user anemotionally-contextual advertisement.

Example 10. Basketball Use Case

The work process of a method in embodiments can be further illustratedthrough a use case example. In this example, the basketball game isbeing analyzed. For simplicity's sake, it can be further assumed thatonly the following entities exist—“team”, “player” and “ball.” Thefollowing connections exist (among others): (a) “shooting a hoop from Xfeet”—a connection between player and ball (multiple instances of thisconnection exist, one for each possible distance); (b) “Has the ball”—aconnection between a player and itself; and/or (c) “Blocked by playerY”— a connection between two different player entities

Table 3 below illustrates the connections among entities of EXAMPLE 7.

TABLE 3 Entity_1 Entity_2 Connection Player Ball Has_ball PlayerBlock_by Ball Shooting_hoop_from_5_feet Ball Shooting_hoop_from_10_feetBall Shooting_hoop_from_15_feet Ball Shooting_hoop_from_20_feet

To assure high initial accuracy and to minimize generation of irrelevantquestions, it may be desirous to train the model before operating it ona live event in real time. This can be done in any number of simulationscenarios, including by allowing a group of users to watch apre-recorded past event and to capture their observations andpredictions for each point of time within the pre-recorded past event,or to input a prior transcript of the event into the model's trainingprocess. Before the training phase event begins, a default time intervalis defined (performed when the function is activated, in line 1000 ofTable 15). This time interval (for example, 2 seconds) will dictate thefrequency for which each group of users will be chosen and queriedeither about events that have occurred or about their predictions forthe future (lines 1020 and 1030 of Table 15). However, this interval mayeither increase or decrease based on the model's assessment of thecurrent state—while a time out is called, for example, the interval maybe increased to 30 seconds in order not to annoy the set(s) of usersengaged by the system, yet still allow some observations to be captured,which can be especially important in order to determine the exact timeat which the timeout ends and normal play resumes. The current state isupdated after every iteration (line 1045 of Table 15) in order tooptimally assess what the time interval should be.

In embodiments of the present invention, the querying process works asfollows: for the point of time the users are to be queried about (atleast once for the past and at least for the future, the process beginsby selecting which the states the users are to be queried about (line1120 of Table 16). For example, an embodiment of the present inventionmay iterate through all possible states in the state transition tableand select the top 50 possibilities based on their statisticallikelihood of occurring, although other embodiments may use othermethods. Next, analysis of the set of answers (about the past) andprediction (about the future) that were obtained in previous queryingrounds is performed. The analysis of the sequence and the selection ofthe states can be performed by the Hidden Markov Models (HMM), describedin later paragraphs below, or by other means. Based on the state of theevent that is deduced to be in at the time (for example, “group A hasthe ball, the ball is 30 feet from the hoop, player X has the ball”).Examples of possible questions are presented in Table 14. It can beappreciated by a person of ordinary skill in the art that functions canbe included in the process to iterate through the possible states andconnections of an event at any point of time and can further utilizeNatural Language Generation techniques, including, for example, withoutlimitation, the SimpleNLG method (A. Gatt and E. Reiter (2009);SimpleNLG: A realisation engine for practical applications. Proceedingsof ENLG-2009; herein incorporated by reference in its entirety) tocreate human-readable questions based on the various possibilities foreach future state to occur. It should be further appreciated that suchquestions can be sent to users via standard HTTP or other IP-basedconnections to either a standard web browser used by the user and/or toany number of connected proprietary applications employed by anembodiment of the present invention.

The training phase begins by calling the “main” function, which theprocess initializes with two parameters: a) the event parameter, whichcontains information about the event its entities, the connectionbetween them and all other relevant attributes; b) the time interval,which provides the default interval for an iteration in the system(within an iteration, sets of questions are sent to users to users andthe users' responses are processed).

The process begins by extracting the Current State: the possible initialstate(s) of the event and assigning them to a dedicated variable (line1005). This is done in order to better enable the HMM process toidentify the likely states (by providing an initial set from which todeduce the observation about which the users will be queried). In abasketball game, for example, the initial state would be{Team_X_on_the_offensive, Team_Y_on_The_Offensive}.

Once the initial states have been identified, the iterative questioningof the users and the analysis of their responses begins. Every timeinterval t (whose frequency is determined both by the default timeinterval and by the current state the model is presumed to be in at themoment), the following process takes place:

First, obtain the users' predictions about the future and impressionsabout the past (lines 1020, 1030 of Table 15). This process is identicalin both cases (except for the time t for which it is executed) andtherefore it is described once here in general.

This function begins by initializing the object that will contain thepredictions made by the users (line 1110 of Table 16).

Then, the States object (which contains all the possible statesaccording to the definitions of the event) is initialized and populated.The possible states are the states that are reachable from the currentpresumed state (that is, the transition probability is greater thanzero).

The next step consists of generating the User Groups Matrix: the groupsof users that will be assigned to each state (line 1130 of Table 16).The number of groups equals the number of possible states (shown inTable 4).

Table 4 below illustrates the segmentations of questions to differentgroups.

TABLE 4 Group Assignments Observation Queried Group_1 Player X shot from15 feet Group_2 Player X shot from 25 feet Group_3 Player Y shot from 15feet . . . . . .

Following the creation of the groups, the process createsQuestionnaires: the various questionnaires that will be used by thevarious groups (line 1140 of Table 16). The questions may be direct(“did player X score 3 points?”), implied (“Did player X dance when hescored 3 points?”) or negative (“did player X miss the basket?”), butall questions, after data analysis, will enable deduction whether or notthe said event took place.

Once this has been completed, for each group the following (starting atline 1160 of Table 16) steps are performed:

First, obtain the details of all users who are available for querying(line 1410 of Table 19). Then, analyze the number of available users andprioritize the possible states in order to determine which of them willbe analyzed if the number of users is not sufficient to analyze them all(line 1420 of Table 19).

Following that, attempt to determine how dynamic the event is at themoment (that is, how likely is the next iteration to arrive). Thisaction (line 1430 of Table 19) determines how long to wait for theanswers to arrive.

The next step is to assign users to each group. This is executed by thefunction Assign_Queries_To_Users, called in line 1440 of Table 19. Theresult of this function is a matrix containing the assignments of usersto each group. An example of this is presented in Table 5.

Table 5 below illustrates an assignment of users to query groups,further recording the time each query was sent to each user.

TABLE 5 Users Group_Assignment Transmission Time User_1 Group_1 T User_1Group_1 T User_1 Group_1 T User_1 Group_2 T + 1 User_1 Group_2 T + 1User_1 Group_2 T + 1 . . . . . . . . .

Once the users have been assigned, the questionnaires are sent and thesystem waits for responses (line 1450 of Table 19).

When the responses are obtained, they are translated into a binaryrepresentation indicating whether the user replied that the state he wasasked about took place (1) or not (0). Once these answers are gathered,the matrix in Table 5 above is populated with two additional columns(shown in Table 6), recording the responses and their respective arrivaltimes

Table 6 below illustrates an assignment of users to query groups,further recording the time each query was sent to each user and the timewhen the response was received.

TABLE 6 Transmission Response Users Group_Assignment Time Responses TimeUser_1 Group_1 T 1 T + 30 sec User_1 Group_1 T 0 T + 25 sec User_1Group_1 T 1 T + 12 sec User_1 Group_2 T + 1 0 T + 30 sec User_1 Group_2T + 1 1 T + 34 sec User_1 Group_2 T + 1 1 T + 33 sec . . . . . . . . . .. . . . .

Once a sufficient number of responses have been gathered, the results(line 1480 of Table 19), are processed to remove responses that appearto be fraud. Since there are “yes” and “no” answers for the occurrenceof each event, it is now possible to analyze them and calculate theprobability of each event (see Table 7).

Table 7 below illustrates the observation probabilities derived fromuser responses to queries.

TABLE 7 Group Observation Yes Answers No Answers Probability Group_1Player X shot 900 100 0.9 from 15 feet Group_2 Player X shot 600 400 0.6from 25 feet Group_3 Player Y shot 300 700 0.3 from 15 feet . . . . . .. . . . . . . . .

These probabilities are returned in the object Analyzed_Responses.

Once the predictions about the future and the impressions about the pasthave been obtained, the process combines them in order to calculate theoverall probability of each observation. This is done in line 1040 ofTable 15, in which the process combines the two sets of probabilitiesinto one (add the right figure with the circles). Once this is done,each observation is assigned with a single probability of its occurrence(see Table 8.

Table 8 below illustrates the collated prediction probabilities.

TABLE 8 Player X Player X Player Y Player Y No one shot from shot fromshot from shot from shot to the 15 feet 25 feet 15 feet 25 feet hoopPrediction 0.9 0.6 0.3 0.2 0.1

When the probabilities in Table 8 are normalized to one, theprobabilities in Table 9 are obtained.

Table 9 below illustrates the normalized prediction probabilities fromTable 8.

TABLE 9 Player X Player X Player Y Player Y No one shot from shot fromshot from shot from shot to the 15 feet 25 feet 15 feet 25 feet hoopPrediction 0.428571 0.285714 0.142857 0.095238 0.047619 (normalized)

After the probabilities are calculated, they can be used to determinewhat the most likely next state is, and the process iterates again forthe next time interval.

Once the event has been concluded, a sequence of observations and stateshas been obtained. Using these, it's possible to generate theState_Transition_Matrix and Observations_Matrix objects (lines 1070 and1080 of Table 15).

Table 10 below illustrates an example of an Observation Matrix.

TABLE 10 Player Y Player Z committed committed foul on foul on Ballenters Ball misses State/Observation Player X Player X hoop hoopPlayer_X_Shooting_hoop_from_5_feet 0.05 0 0.5 0.37Player_X_Shooting_hoop_from_10_feet 0.08 0.08 0.4 0.44Player_X_Shooting_hoop_from_15_feet 0.1 0 0.35 0.55

Table 11 below illustrates an example of a States Transition Matrix.

TABLE 11 Team X Player 1 Player 2 Player 3 Player 4 is on ShootingShooting Shooting Shooting the hoop hoop hoop hoop offen- from 5 from 5from 5 from 5 sive feet feet feet feet . . . Team X 0.1  0.05 0.06 0.040.05 . . . is on the offensive Player 1 0.02 0.1 0.08 0.1 0.04 . . .Shooting hoop from 5 feet Player 2 0.06 0.02 0.1 0.07 0.07 . . .Shooting hoop from 5 feet Player 3 0.06 0.04 0.05 0.1 0.04 . . .Shooting hoop from 5 feet Player 4 0.07 0.1 0.05 0.04 0.1 . . . Shootinghoop from 5 feet • • • • • • . . . • • • • • • • • • • • •

The final step of the training phase is the creation of the affinitymatrix (using the Affinity_Matrix object). This is a user-entity matrix,where entries in specific user-entity cells indicate that the user has acertain sentiment towards the entity. A numeric values is used torepresent the strength of the connection. It should be noted that forsome users, the some of the cells in the matrix will remain empty.

Table 12 below illustrates an example of an Affinity Matrix.

TABLE 12 Entity_1 Entity_2 Entity_3 Entity_4 Entity_5 . . . User_1 2 4 .. . User_2 5 . . . User_3 4 . . . • • • • • • . . . • • • • • • . . . •• • • • • . . .

The Runtime Phase

The running phase is initialized with the following parameters: theevent (with the same attributes as those of the training phase), thedefault time interval (can be the same as was used in the trainingphase, but there is no such requirement) and the two matrices which areamong the products of the training phase—the states transitions matrixand the observations matrix.

The runtime phase begins by obtaining the initial state(s) of theevent—as was done in the training phase (line 2005 of Table 22).

Then, for each time interval, may do the following: (a) Get the userpredictions for the past and the future (lines 2020 and 2022 of Table22), combine and normalize them (lines 2024 and 2030 of Table 22);and/or (b) Generate a temporary matrix that contains the probabilitiesof the observations made by the user for the current iteration (line2040 of Table 22). This temporary matrix may contain the following.

Table 13 below illustrates a temporary matrix containing the userobservation probabilities.

TABLE 13 Player X Player X Player Y Player Y No one shot from shot fromshot from shot from shot to the 15 feet 25 feet 15 feet 25 feet hoopsPrediction 0.428571 0.285714 0.142857 0.095238 0.047619 (normalized)

-   -   a) Update the states sequence based on the new information        (e.g., with Hidden Markov Models using the Baum-Welch algorithm        or any of its derivatives, Baggenstoss, Paul M. “A modified        Baum-Welch algorithm for hidden Markov models with multiple        observation spaces.” Speech and Audio Processing, IEEE        Transactions on 9.4 (2001): 411-416.)    -   b) Present commercial content to the users, based on the current        events and their affinity matrix values.    -   c) Update the user-entity affinity matrix. This matrix is the        same one that was generated in the training phase, but        additional values will be added to it as more and more        information is collected from and about the users, due to the        continuous interaction with them.    -   d) The number of groups is dependent both on the number of        available users (spread over the time intervals) and that of the        possible states. For demonstration purposes, a simple allocation        is applied, designed to ensure that the responses the system        obtains are certain to be true with a statistical significance        of 90% (assuming a normal distribution in user responses),        however one skilled in the art can appreciate that other methods        of group segmentation can be realized in one or more embodiments        of the present invention, and all such methods are incorporated        herein in their entirety. For example purposes, the method only        queries users whom the process deems “trustworthy” (users whose        past responses have accurately coincided with actual data        gathered during past events or within a past timeframe of the        current event).    -   e) Once the groups have been generated, questions are allocated        to each group (line 1140 of Table 16). The questions are derived        from one state of the set of possible ones the system has        previously generated. The questions can be derived automatically        (for example, one possible question for the state “group A has        the ball, the ball is 30 feet from the hoop, player X has the        ball” is “did player X drop the ball 30 feet from the hoop?”),        and multiple questions can (and are likely to) be generated for        each state. Questions can also come from other users or from        cohorts of the system (human or mechanical). Answers can come        from the body of users or from cohorts of the system (human or        mechanical). In addition to computing user reliability, external        data can be used to arbitrarily define the reliability of any        user, group or cohort, for example to enable certain trusted        users to be assigned a high reliability ranks, for example if an        expert basketball referee would become an employee of the        operators of an embodiment of the present invention, and would        then provide human-tagged input, optionally via the form of past        impressions, for which the system will fix their probability and        credibility as, for example, 100%. Similarly, external data        which is known to be accurate can be incorporated into the        process as past impressions that with a fixed probability and        credibility, for example, of 100%.

Table 14 below illustrates exemplary questions (both on past and futureevents) presented to users.

TABLE 14 Question State Type Question TextPlayer_X_Shooting_hoop_from_5_feet Past Did player X shoot a hoop?Player_X_Shooting_hoop_from_5_feet Past Was a shot made from X feet?Player_X_Shooting_hoop_from_5_feet Past Was the shot successfulPlayer_X_Shooting_hoop_from_5_feet Future Will a “time out” be called?Player_X_Shooting_hoop_from_5_feet Future Will Player X shoot a hoopagain in the next minute? Player_X_Shooting_hoop_from_5_feet FutureWhich player will be MVP?

Once the groups and queries are created, each group is assigned a query(line 1160 of Table 16) and query is sent to all the group's members.Once the queries are sent, the process waits for an answer from theusers. The span of time the process may wait for each user to responddepends on a set of factors: for example, preferably the user'sreliability metric (past on past interactions), more preferably also theuser's location (determined by GPS, user's claims, connection metadata,phone metadata, audio content recognition and other signals as statedherein and otherwise found in the industry), and most preferably themedium through which the user is likely to obtain information about theevent and so forth. Different embodiments may opt for any mixture ofparameters, and that a person of ordinary skill in the art willappreciate that any number of such parameters can be collected and usedby an embodiment for this purpose.

Once the number of obtained answers reaches a satisfactory number—onethat enables the method to obtain statistical significance—the systemcalculates the probabilities of the perceived events (Table 16, line1170). For example, if 7000 users indicated the basketball shot was “in”and 3000 said it was “out”, the probability assigned to the formerobservation is 70% and the probability assigned to the latter is 30%(for ease of reference, the term “observation” is used in accordancewith the terminology used by Hidden Markov Models as previously citedherein and further described below). One can appreciate that HiddenMarkov Models are merely one possibility for the sequencing algorithmand that other methods are available and will become available, and thescope of the present invention is intended to not be limited to onlythat sequence algorithm described for this example.

Once this iterative loop process of Function 1100 shown in Table 16 iscompleted, the observations made by the groups of users being iteratedcurrently are added to a “container” object (Table 16, line 1180) andonce all groups are queried this container is returned to the parentfunction (Table 16, line 1190). Note that this process can bedistributed to a plurality of computing devices, optionally in differentgeographic regions, which perform this operation in parallel to allowanalysis of a very large number of users and/or groups in a very smallamount of time.

Once the predictions (about the future) and impressions (about the past)for the entire event are obtained, the event is over, and all the rawdata are available), then it is time to combine them (Table 15, line1040). In one embodiment, this can be done by assigning, for example, a2:1 ratio to past observations versus future predictions. Obviously,more advanced methods can be applied, but at one or more embodiments canuse this approach. This process is presented in FIG. 4 . Should the needarise, the model is capable of functioning even when only partialinformation about the event is available. In embodiments the process maybe described utilizing the entire event.

Referring to FIG. 4 , a generic explanation of the process by which pastimpressions, future predictions and final observations are determined bythe system. In this example, past impressions relating to t−1 401collected from users at time t are combined with future predictionscollected from the users at the same time t, asking the users what theypredict will happen at time t+1 402 to create the final stateobservations 403 that will be used by the HMM. The probabilities of eachobservation are calculated separately for the past impressions and thefuture predictions. Once the probabilities are calculated, they areintegrated into a final observation. For ease of reference, the methodused for combining the probabilities is a weighted averaging techniquewhere the weight assigned to past impressions is by default twice thatof future predictions. This ratio may be modified, in embodiments, forexample based on user reliability, response latency, external data (suchas sensory data from one's phone phone) and/or other external factors,to name a few. Continuing the example, a first past impression, PI-1,can be matched with a second future prediction, FP-2. Once more data isprocessed by the HMM, resultant Final Observation FO-2 is deemed to havethe highest statistical probability that FO-2 has actually occurred.

Once a unified set of observations is created, it is possible togenerate the observation states which are an integral part of the HiddenMarkov Model (Table 15, line 1050). This is done in the followingmanner: for each pair of entities (“player and ball” or “player1 andplayer2”), the method checks whether a connection between them is deemedpossible by the definition of the event (Table 17, line 1230), as it wasspecified during the definition of the event. If the connection ispossible and the input provided the users support its existence, thenthe program creates the observations state that represents theconnections and add it to a list of observation states (line 1240 ofTable 17). Once all entity combinations have been analyzed, the set ofobservation states is returned (Table 17, line 1250).

Once all the observations are obtained, the system can generate thestates transition matrix (Table 15, line 1060). This matrix can bedefined by a set of domain experts, and it defines all the “legal”transition between states in the event “eco system.”

Example 11. State Transitions

A further example will assist describing the concept of statetransitions in accordance with one or more embodiments of the presentinvention: during the above-mentioned basketball game, it is possible totransition from a state of “foul committed” to a state of “penaltyshot.” This state, in turn, can transition to another “penalty shot” orto “Player X has the ball.” An illegal transition would be from “playerX from team 1 has the ball” to “player Y from team 2 shooting hoop from30 feet”; a state such as “Player X loses ball to player Y” must firsttake place. An example of the possible state transitions is presented inFIG. 5 .

Referring to FIG. 5 , a chain of events at a point in time t can bedepicted, for example, one analyzing an exemplary basketball game.Sequence 501-505 show the sequence of transitions of the state of thegame from one to the next, in this case with team Y being on theoffensive 501, player 6 (inferred to be from team Y) shooting the ballfrom 10 feet 502, followed by team Y's score being incremented by two503, thereby inferring that a basket had been made. Next, the system caninfer that team X is now in possession of the ball, as it is on theoffensive per 504 and player 1 (inferred to be from team X) shoots theball from five feet 505 at time t.

As stated, at this point in time, the system begins another iteration ofthe algorithm (as depicted in line 2010, Table 22). Once again, the timeinterval between two subsequent runs is determined by the time intervalset by the modeling addition to the system's assessment of the currentstate, as defined in line 2075 of Table 22. Based on the current stateof the state transition table, (“player 1 shoots from 5 feet”) thesystem generate queries regarding the chances of the shot succeeding(the future) and regarding the distance from which the shot was thrown(the past). This is described in line 2020 of Table 22.

Once a sufficient number of answers has been obtained, an embodimentprovides for normalizing the probabilities of all observations to one(line 2030 of Table 22) and then generating the top likely observations(line 2040 of Table 22). Using these observations, the method can usethe matrices presented above to update a perceived chain of events (line2050 of Table 22).

For simplicity of this example, it can be assumed that there is only onelikely observation—that player 1 shot 3 points. This conclusion isreached following the analysis of past impressions and futurepredictions, as depicted in FIG. 4 Given this observation (and sincethere is zero probability that a shot from 5 feet could result in 3points), the method can update its model and conclude (after reassessingall state transition probabilities) that player 1 shot the ball from atleast 23 feet 9 inches (2013 NBA rules), as defined in the statetransition diagram 201 and depicted in FIG. 2 . As a result, the systemupdates the chain of events, as shown in FIG. 6 :

Referring to FIG. 6 , a revised chain of events at a point in time t+1(relative to time t of FIG. 5 ) can be depicted in accordance with oneor more embodiments, analyzing an exemplary basketball game. Sequence601-604 is the same as sequence 501-504 (respectively) of FIG. 8 , showthe sequence of transitions of the state of the game from one to thenext, in this case with team Y being on the offensive in step 601,player 6 (inferred to be from team Y) shooting the ball from 10 feet instep 602, followed by team Y's score being incremented by two in step503, thereby inferring that a basket had been made. As mentioned in step504 the system can infer that team X had possession of the ball, as itwas on the offensive and player 1 (inferred to be from team X) wasbelieved to have shot the ball from five feet in step 505. However, newobservations have since arrived, suggesting that team X's score wasincremented by 3 points, mandating an update to the sequence andincreasing the distance thrown from 5 feet to at least 23 feet 9 inches,depending on the probabilities defined in the state transition table. Asthe system iterates through time t+2 and onwards, the actual distancewill be derived by analyzing ensuing observations as well.

Finally, the method generates the final component needed for the HMMprocess—the observation matrix. This matrix denotes the likelihood ofobtaining a certain observation at a certain state, thus enabling theprobabilistic process of the HMM to “interpret” the observationsobtained from a set of users. The process used to generate this matrixis called in line 1070 of Table 15.

In embodiments, the process used to generate this matrix can be asfollows: for each state (line 1320 of Table 18) the method analyzes eachobservation (line 1330 of Table 18) and checks whether the observationis possible (line 1340 of Table 18), as was defined so by the expertswho set the parameters of the event prior to the experiment. If theobservation is “legal”, then the system assigns it to the said statewith its probability (line 1350 of Table 18), as was determined duringthe analysis of the users' responses. This process results in a table(or matrix) where for each state/observation combination, a valuerepresenting its likelihood is present. An example of such a datastructure is presented in Table 11.

This process is repeated iteratively throughout the course of the event.It should be noted that the present invention is by no means limited toonly modifying the latest state in the chain: additional informationfrom users with higher latency will also be taken into account (based onreliability and fraud filtering) and may very well be used to update theevents. For example, in the case of a shot that was later disqualifiedby a referee only the responses received after the referee's call willcontain “correct” observations. By maintaining and updating allobservations and probabilities (for any amount of time) the method canaddress this issue.

In embodiments the system may utilize this information for commercialpurposes. This is done in the following way: using the affinity matrix(generated in the training phase, if available) and the current affinityof the user to entities involved in the current state of the eventschain (line 2060 of Table 22), the system attempts to determine whetherthe user has a strong positive affinity to any of the relevant entities.By iterating over all possible entities (for every user), it is possibleto identify the entities for which the user has the greatest positiveaffinity (lines 2210-2240 of Table 24) and then either generate and sendrelevant advertisements, promotions, or other similar material, and/orsignal an external system with this data for any action that system maythen take. Related methods of targeted advertisement can use existingmethods in the field of Recommender Systems presented by academia andothers. The methods that can be used for identifying entities for whichusers have high affinity include collaborative filtering (Koren, Yehuda,and Robert Bell. “Advances in collaborative filtering.” In RecommenderSystems Handbook, pp. 145-186. Springer US, 2011.) and matrixfactorization techniques including SVD (Weng, Xiaoqing, and Junyi Shen.“Classification of multivariate time series using two-dimensionalsingular value decomposition.” Knowledge-Based Systems 21, no. 7 (2008):535-539), both of which are incorporated herein by reference in theirentirety.

FIG. 7 depicts an exemplary computer function for the training phase ofthe proposed method. Processing group 715 includes input data includingevent information and an initial time interval. Main processing group716 includes the event state query step 703, determines the currentevent state, followed by loop 704 which iterates through each occurrencewithin a series of occurrences that include an event. Actual splitsbetween occurrences can be time based, rule based, play based, externalinput (human, data or otherwise, including wireless signals, GPSlocation and time, microphone input, social media connections,interactions and check-ins, information prompts responses, withoutlimitation) or otherwise, or any combination thereof, withoutlimitation. Step 705 and 706 may run in parallel, sending questions tousers and receiving the user responses by calling the function depictedin FIG. 8 twice: once for future predictions in step 705 that werecaptured prior to the selected occurrence of the same event (or usingdata from past events, or any combination thereof) for each of thequestions about said selected occurrence, and a second time to receivepast impressions in step 706 that were captured after said selectedoccurrence occurred (following the selected occurrence of the same eventor using data from other events, or any combination thereof, withoutlimitation) for each of the questions about said selected occurrence.Then, step 707 combines the results of both the future predictions fromstep 705 and past impressions from step 706 that were collected into aunified User Predictions Matrix which is then filtered by step 708 bycalling the function on FIG. 17 , optionally using external data sources711 which may include human or machine generated data, includingwireless signals, GPS location and time, microphone input, social mediaconnections, interactions and check-ins, information prompts responses,without limitation. Step 709 then updates the current Event State andObservation Matrices, assessing most probable State Transitions andselecting the smallest time interval allows the method not to miss anyoccurrence yet not create excessive questions to users, prior to step710 looping back to step 704 to process the next occurrence. Afterprocessing all event occurrences, step 712 combines the future and pastuser predictions captured from the users and updates the StateTransition Matrix and the Observation Matrix, containing a possible amatrix of states (combinations of entities and occurrences whose chanceof happening is greater than a threshold), by calling the function onFIG. 19 . One ordinarily skilled in the art will appreciate that, inembodiments, the system may then provide for human or external input tobe used to further refine the data in step 713, for example using the asupervised learning method such as “The Wekinator” (Fiebrink, R., P. R.Cook, and D. Trueman. “Human model evaluation in interactive supervisedlearning.” Proceedings of the SIGCHI Conference on Human-ComputerInteraction (CHI' 11), Vancouver, BC, May 7-12, 2011.). Finally, theUser Affinity Matrix is generated in step 714 by calling the function inFIG. 13 .

Table 15 below illustrates an exemplary pseudo code implementation of amain program routine for the training aspect, as applicable to one ormore embodiments of the present invention.

TABLE 15 Line (FIG./Step) Comment Pseudo Code 1000 Main program routinefor Main(event, time_interval) (7/716) the training phase. Assumes aState 1005 The current state of theCurrent_State<-event.Get_Initial_State( ) (7/703) event is required sothat it would be possible to better assess what the next time intervalshould be. Initially, the state is defined by the type of the event.1010 Starts a loop that iterates Foreach (t inSplit_To_Time_Intervals(event.length, (7/704) through each occurrenceCurrent_State)) within a series of occurrences that include an event.Actual splits can be time based, rule based, play based, external input(human, data or otherwise) or otherwise, and any combination thereof.1020 For each occurrence of the   User_predictions_1[t] <- (7/705)selected event, call Generate_User_Predictions_For_Time_Interval(t-1,function 1100 (FIG. 8) to event, False) read user predications that werepreviously captured (prior to the selected occurrence of the same eventor using data from past events, or any combination thereof) for each ofthe questions about said selected occurrence. 1040 Next, the two typesof   User_predictions[t] <- (7/707) predictions (past andCombine_Predictions(User_predictions_1, future) are combined intoUser_Predicitons_2) one matrix and normalized, with the resultillustrated in Table 13. 1041 Filter non-credible users  Calculate_User_Response_Time_Fraud_Like- (7/708) by calling function1700 lyhood(Response_Time, Event_Dynamic_Level, (FIG. 17). User_Details)1045 Now, update the current   Current_State<- (7/709) state. Assessseveral likely Determine_Current_State(User_predictions[t]) options andchoose the smallest time interval that fits one of them, reducing thechance of missing anything, while also not bothering users with extraand redundant questions. 1048 End For (7/710) 1050 Next, combine theuser Observation_States<- Generate_Entity_Connections (7/712)predictions captured from (event, User_Predictions) the users in timeinterval t as depicted in FIG. 4 and generate possible a matrix of“states” (combinations of entities whose chance of happening is greaterthan the threshold) by calling function 1200 (FIG. 19). 1060 Next,optionally employ State_Transition_Matrix<- (7/712) external data(captured by Generate_State_Transition_Matrix(event) human or mechanicalAn example of the matrix is presented in Table 11. means) to furtheranalyze the results, and or use any combination of external data sourcesto remove any invalid states, thereby improving accuracy of thetransition matrix. 1070 Finally, create an Observations_Matrix<- (7/712)observation matrix using Calculate_Observation_Matrix(event, the statetransition matrix Observation_States, State_Transition_Matrix) and thevarious observations that were collected, as illustrated in Table 10.1075 After training, optionally (Not illustrated, can use any taggingand supervised (7/713) pass all information to learning method e.g.Fiebrink, R., P. R. Cook, and D. human taggers for final Trueman. “Humanmodel evaluation in interactive tagging and analysis. supervisedlearning.” Proceedings of the SIGCHI Conference on Human-ComputerInteraction (CHI' 11), herein incorporated by reference) Vancouver, BC,May 7-12, 2011.) 1080 Once the states of the Affinity_Matrix<- (7/714)event have been set, it'sGenerate_Users_Affinity_Matrix(State_Transition_Matrix, time to assessthe user's Observations_Matrix, User_predictions) affinity (fondness)for each entity. For example, if the user interpreted the facts in amore favorable manner (for an entity) than the probabilities wouldotherwise suggest, or if the user's predictions were more optimisticthan reality, the system can deduce his affinity alignment. A sampleAffinity Matrix is illustrated in Table 12. End Main Routine

FIG. 8 represents a computer function to generate user predictions for aselected time of an event. Processing group 814 includes input datawhich is used to provide source data for the function, includinginformation about event data, the selected time, a flag to indicate ifpast impressions or future predictions are to be processed, and theminimum user credibility score that should be used. Processing group 815denotes the main execution code, starting with step 804, in which theUser Predictions Matrix is initialized. Then, in step 805 the EventState Matrix is filtered to remove any illegal entries (for example, inNFL football, an “incomplete rush” is not a valid state and will thus beremoved). Next, users are segmented into groups in step 806, for exampleby determining that as many groups as needed will be created to enableeach possible event state option to be sent to a minimum of 50 usersthat each have a credibility score over 0.8, and optionally usingadditional data including user reliability data or other data sources.Next, in step 807 a matrix of possible (and legal) questions isgenerated; these questions relate to each of the possible states of theevent based on the Event State and Event Observation matrices and thusthe model's anticipation of what might happen. Then, iteration loop 808begins iterating all groups created in step 806, sending questions toall users in step 809 (by calling the function in FIG. 11 ), for examplevia transmission over a communication network as depicted in FIG. 19step 1910. Note that the parameters controlling the order anddistribution of questions to users may also include latency data of eachgroup 812, which is the system's anticipated latency (based on, forexample user phone metadata and past performance) of how quickly membersof the group will respond. Upon receipt of user responses, theprobabilities are normalized to one in step 810, followed by step 811 inwhich the normalized user responses are added to the User PredictionMatrix and the iteration cycle is repeated. Finally, step 813 returnsthe User Prediction Matrix to the calling function.

Table 16 below illustrates an exemplary pseudo code implementation of aprogram function to capture user predictions based on the number ofpossible likely observations, as applicable to one or more embodimentsof the present invention.

TABLE 16 Line (FIG./Step) Comment Pseudo Code 1100 Program function toGenerate_User_Predictions_For_Time_Interval (8/815) capture userpredictions (t, event, is_prediction_about_future, based on the numberof Min_Credibility_Score) possible likely observations (based on pastobservations, user data, event state and decision graph, determine thenumber of makeup of the population to query. 1110 Initialize UserPredictions User_Predictions<-(init) (8/804) Matrix. 1120 Then use statediagram to States<-Determine_Possible_States(t,event) (8/805) determineALL possible states for event (for example, in NFL football, an“incomplete rush” is not a valid state) as illustrated in FIG. 2. 1130Next, segment active users User_Groups<-Determine_Users_Groups(t,event,(8/806) and determine how many Min_Credibility_Score) groups will begenerated, optionally requiring that the users each of those groups themhave a high credibility score. Segmentation can include location datafrom phone or any other source. A sample result of this step isillustrated in Table 5. 1140 Next, generate matrix ofQuestionnaires<-Generate_Queries(t,event) (8/807) possible (and legal)questions that relate to each of the possible states of the event basedon the Event Observation Matrix and thus the prediction of probabilitiesas to what might happen. 1150 Start iterating through Foreach (group inUser_Groups) (8/808) each group in the list of groups created in line1130. 1160 Call function 1400 (FIG.   Answers<- (8/809) 11) to sendquestions to Assign_Questionnaires_To_User_And_Query(Question- users andwait for naires,group,event, t, States, group .latency, responses. Notethat the is_prediction_about_future) latency of each group is also used,as a - a prediction (based on phone metadata and past performance) howquickly members of the group will respond. A sample result of this stepis illustrated in Table 5. 1170 Normalize answers  Answers<-Normalize_Answers(Answers) (8/810) received from userresponses. 1180 Add user responses from   User_Predicitons.Add(Answers)(8/811) data collected in previous step. End For 1190 Output matrix ofuser Return User_Predictions (8/813) predictions. The result of thisfunction is illustrated in Table 9.

FIG. 9 represents a computer function to generate valid connectionsbetween various entities of an event (for example, ball is connected toplayer, player is connected to team, dress is connected to actress, blueis connected to dress, etc.), which will be populated into the UserPredictions Matrix in anticipation of derived questions being sent tousers. Processing group 909 includes input data including informationabout event data and the User Predictions Matrix. Processing group 910denotes the actual execution code, starting with step 904 whichinitializes the User Predictions State Matrix to initially contain allpossible valid entity connection using data derived from StateTransition Diagram 201, for example, and using probabilities that werederived from analysis of past events or by human tagging ofrelationships of entities that appeared in past events. Then, step 905begins an iteration loop which in step 906 evaluates each entity inrelation to every other entity, and when a connection between twoentities is found in any prior prediction (for example, from the UserPredictions Matrix 903) or observation (including data from the trainingphase), step 907 adds the connection and its probability to the UserPredictions Matrix which is finally returned to the calling function instep 908.

Table 17 below illustrates an exemplary pseudo code implementation of aprogram function used to generate the states based on the connectionsbetween entities of an event, as applicable to a one or more embodimentsof the present invention.

TABLE 17 1200 Program function used to Generate_Entity_Connections(event, (9/ generate the states based User_Predictions) 910) on theconnections between the entities of an event. 1210 The possible list ofUser_Prediction_States<-(new) (9/ prediction combinations 904) andlikelihoods for each are obtained (for example, by analyzing a pastevent or by following a decision graph e.g. 201). 1220 Recursivelyiterate all Foreach (entity_type_1 in event.entities) (9/ entity types.905) 1230 Within main iterative   If (9/ loop, evaluate each entity(entity_type_1.equals(entity_type_2) ∥ 906) vs the entity selected in!event.Connection_Exists(entity_type_1, step 1220 to determine,entity_type_2)) then process next item for example, if actress isconnected to dress and then if dress is connected to color. 1240 Connectlist entities for   User_Prediction_States<- (9/ which any relationshipConnect_Entities_Based_On_Predic- 907) was found.tions(entity_type_1.items, entity_type_2.items, User_Predictions) EndFor 1250 Return list of possible Return User_Prediction_States (9/states to be offered to 908) users.

FIG. 10 represents a computer function to calculate the probability ofany particular observation to occur in any particular event state. Thesevalues are to be placed in a matrix object, and enable the use HiddenMarkov Models (or other analysis methods, without limitation) to inferthe true sequence of event elements in the real world. Processing group1011 includes input data, including event data and the currentObservation States and State Transition matrices. Within main executiongroup 1012, Step 1004 initializes the Observation Matrix, followed bytwo nested iterative loops 1005 and 1006, respectively. Outer loop 1005iterates through all states, and inner loop 1006 iterates allobservations within the state currently evaluated in step 1005. Step1007 evaluates the possibility of each observation within occurringwithin the iterated state, continuing to step 1008 if a possibleconnection exists or returning to step 1006 if not, in which case thenext possible observation is evaluated. If a possible connection exists,step 1008 updates the Observation Matrix to connect the iterated stateto the iterated observation using the probability derived from the UserPrediction Matrix. Step 1009 returns execution to the relevant iterativeloop, once all states and all observations within have been evaluated,step 1010 returns the Observation Matrix (illustrated above in Table 13)to the calling function.

Table 18 below illustrates an exemplary pseudo code implementation of aprogram function used to calculate the probabilities of a particularobservation to occur in a particular state, as applicable to one or moreembodiments of the present invention.

TABLE 18 Line (FIG./ Step) Comment Pseudo Code 1300 Based on the statesand Calculate_Observation_Matrix (10/ their assigned (event, 1012)probabilities, calculate the Observation_States, probabilities ofState_Transition_Matrix) observation O to occur in state S. these valueswill be placed in a matrix object, and enable the use of Hidden MarkovModels to infer the true series of events in the real world. 1310Initialize observation Observation_Matrix<-(new) (10/ matrix. 1004) 1320Main iterative look that   Foreach (10/ evaluates each state (State inState_Transition_Matrix) 1005) within the state transition matrix. 1330Within main iterative     Foreach (Observation in (10/ loop, furtherrecursively Observation_States) 1006) iterate each observation in theobservation states matrix 1340 Evaluate if an observation If(event.Observa- (10/ selected in step 1330 istion_Is_Possible_In_State(State, 1007) possible in the stateObservation) selected in line 1320. 1350 Then add the observation      Then (10/ selected in line 1330 as a Observation_Matrix.Add(State,1008) valid possibility for the Observation) state selected in line1320. 1351   End for (10/ 1009) 1352 End for (10/ 1009) 1360 (a sampleof such returned Return Observation_Matrix (10/ matrix is illustrated in1010) Table 10)

FIG. 11 represents a computer function to send selected questions toselected users and receive their timestamped responses. Processing group1116 includes input data elements including data elements containing thecurrent Questionnaires and User Prediction States matrices, groupassignment and latency, event data and time, and a flag to determine ifthe generated questions (and ensuing analyzed responses) relate to pastimpressions or future predictions. Within main execution group 1117,step 1106 analyzes the user's characteristics including location,reliability and latency (without limitation), followed by step 1107which obtains the most likely possible states (that is, the valid stateswith the highest probability, including data from the State andObservation matrices, without limitation). Next, step 1108 analyzes therequired sampling frequency of the selected event; for example, afootball game with its 25 second play timers is substantially “faster”than a chess match with a 2 minute play timer, thus requiring a fastersampling frequency. This information is very important for determininggroup partitioning (which users to query), as the latency of the varioususers becomes more and more important the higher the sampling frequency.Step 1109 calls the function on FIG. 12 to assign possible states to thegroups most likely to respond accurately and quickly, followed by theactual transmission of the questions (past impressions or futurepredictions) to the users. It should be noted that more than onequestion can be sent to any group, and that groups may be partitioneddifferently for each question. User responses arrive back to the systemin step 1110, upon which the responses are evaluated for fraud in step1111 (using the function on FIG. 15 ) by analyzing response data foreach user and evaluating if it is within a threshold. The threshold canbe dynamically adjusted depending on various factors including number ofactive users, their locations, connectivity methods, broadcast delays,event sampling frequency as well as external data sources, all withoutlimitation. It should further be noted that for the purpose of thisexample, an exemplary value such as 0.8 can be used, which represents arequirement that the user has provided correct answers at least 80% ofthe time in the past 30 minutes, and that additional embodiments maymake sure of alternate or more advanced methods for fraud detection, allof which are incorporated herein in their entirety. Suspected frauds areflagged in step 1112, while legitimate responses are processed andnormalized in step 1113, followed by removal of fraudulent responses andaggregation of the results in step 1114. Finally, the Analyzed ResponseMatrix is returned to the calling function in step 1115.

Table 19 below illustrates an exemplary pseudo code implementation of aprogram function used to send selected questions to selected users andcollect their timestamped responses, as applicable to one or moreembodiments of the present invention.

TABLE 19 Line (FIG./Step) Comment Pseudo Code 1400 Function to sendselected Assign_Questionnaires_To_User_And_Query (11/1117) questions toselected users (Questionnaires, group, event, t, User_Prediction_States,and collect their group_latency, is_prediction_about_future) timestampedresponses. 1410 Calculates the user'sUser_Details<-Obtain_Users_Characteristics(group, (11/1106)characteristics (location, event, t, User_Prediction_States)reliability, etc.). 1420 Determine which are the Likely_States<-(11/1107) possible states and what isObtain_Likely_States(User_Prediction_States, t, event, the likelihood ofeach, is_prediction_about_future) based on evaluating the data in themodel's matrices. 1430 Factor the “tempo” of an Event_Dynamic_Level<-(11/1108) event into the Determine_Event_Dynamic_Level(event, t,calculations; for example, Likely_states, is_prediction_about_future) afootball game with its 25 second play timers is substantially “faster”than a chess match with a 2 minute play timer. This information is veryimportant for determining group partitioning (which users to query), asthe latency of the various users becomes more and more important theshorter the event timers are. 1440 Based on all these factorsUser_Queries_Assignments<- (11/1109) (among others andAssign_Queries_To_Users(event, Questionnaires, without limitation), thegroup, User_Details, Likely_States, questionnaires can now beEvent_Dynamic_Level, group_latency) generated by calling Function 1500(FIG. 12), with a sample result illustrated in Table 5. 1450 Sendquestions to users Response_Time<-Wait_For_Response( ) (11/1110) basedon assignments generated in steps above and measure their responsetimes. 1460 Call Function 1700 (FIG. If (11/1111) 15), to analyzeresponse (Calculate_User_Response_Time_Fraud_Likelyhood time for eachuser and (Response_Time, Event_Dynamic_Level, User_Details) < evaluateif it is within a Minimum_Legitimacy_Threshold( ) threshold. Thethreshold Note: can be dynamically for the purpose of this example, thefunction adjusted depending on Minimum_Legitimacy_Threshold( ) returnsthe value various factors including 0.8, which represents a requirementthat the user has number of active users, provided correct answers atleast 80% of the time in the their locations, past 30 minutes.connectivity methods, broadcast delays, event dynamics etc. 1470 Returnnull if a users' Return (null) (11/1112) response is consideredfraudulent and is excluded. 1480 Process the responsesResponses<-Process_Query_Responses(group, (11/1113) obtained from theusers, Questionnaires, User_Queries_Assignments) as illustrated in Table7. 1490 Enhance raw responses Analyzed_Responses<- (11/1114) obtained inline 1480 by Remove_Frauds_And_Aggergate_Results(Responses) removingfraudulent responses and aggregate responses. 1495 Return enhancedReturn Analyzed_Responses (11/1115) responses to calling function, asillustrated in Table 9.

FIG. 12 represents a computer function to determine what type ofquestion to send to each user. In embodiment, this is determined by thenumber of available users, the type of the event, the confidence indexof possible events and historical user behavior. Processing group 1212includes input data elements, including event data, Questionnaires, Userand Group matrices, group latency data and the current level of activityin the event. Within main execution group 1213, step 1206 analyzes thereliability of each user, using (for example) the users' details andtheir group assignment. This function is not demonstrated, however, itshould be noted that for the purpose of this example, an exemplary valuesuch as 0.8 can be used, which represents a requirement that the userhas provided correct answers at least 80% of the time in the past 30minutes, and that additional embodiments may make sure of alternate ormore advanced methods for fraud detection, all of which are incorporatedherein in their entirety. In step 1207, the states that the user will bequeried about are chosen. Also, the risk level metric is used todetermine how vulnerable is the current event, in its present state, toattempted fraud: in fast-changing events it is more difficult to quicklydetect fraud than in slower ones. Next, the states the users will bequeried about are chosen in step 1208, followed by ordering of the usersin step 1209 such that the most reliable users are queried first (longlatency also adds to risk, so higher reliability is given to users thatrespond quickly, thereby improving accuracy of the data). The questionsto be sent to the users (in the order determined by step 1209) are thengenerated in step 1210 and finally the User Queries table returned tothe calling function in step 1211.

Table 20 below illustrates an exemplary pseudo code implementation of aprogram function used to determine what type of question to send to eachuser, as applicable to one or more embodiments of the present invention.

TABLE 20 Line (FIG./ Step) Comment Pseudo Code 1500 Program function toAssign_Queries_To_Users(event, Questionnaires, (12/ determine what typeof group, User_Details, Likely_States, 1213) question to send to eachEvent_Dynamic_Level, group_latency) user. This is determined by thenumber of available users, the type of the event, the confidence indexof possible events and the historical user responses. 1510 Evaluate thereliability of User_Reliability<- (12/ each user, using (forDetermine_User_Reliability(event, 1206) example) the users' detailsUser_Details, group_latency) and their group Note: This function is notdemonstrated. assignment. However, for the present purpose, assume thata reliable user is one whose answers have been 80% correct (or more) inthe past 30 minutes. 1520 Choose the states that theRisk_Level<-Determine_Risk_Level (12/ user will be queried about.(Event_Dynamic_Level, event, Likely_States, 1207) Also, the risk levelis used Group_latency, User_Reliability) to determine the vulnerabilityto fraud attempts: in fast-changing events it is more difficult todetect fraud than in slower ones, given all other factors are equal.1530 Populate matrix and Chosen_States<- (12/ choose the states the userSelect_Relevant_States(Likely_States, 1208) will be queried about.User_Reliability, event, Risk_Level) 1540 Users are ordered so thatUser_Order<- (12/ they are queries by orderOrder_Users_By_Risk(User_Reliability, 1209) of the least risky firstRisk_Level, Chosen_States) (long latency also adds to risk, so higherreliability is given to users respond quickly, thereby reducing noise inthe model). 1550 Table of queries for usersUser_Query<-Generate_User_Query (12/ is generated. (event,Chosen_States) 1210) 1560 Return user queries table Return User_Query(12/ to calling function. 1211)

FIG. 13 represents a computer function to create or update an affinitymatrix that stores the predicted sentiment between users and entities,used to determine which entities are liked by each user and to whatdegree, by analysis of user responses. Processing group 1314 includesinput data elements including the Observation, State Transition and UserPredictions matrices, among others and without limitation. Processinggroup 1315 includes the Affinity Matrix initialization step 1305 whichinitializes the Affinity Matrix, followed by four nested iterativeprocessing loops which iterate event intervals 1306, state transitions1307, observations 1308 and finally users 1309. For each combination ofinterval, state transition and observation, the actions of each user areanalyzed. For example purposed, a simple increment of observation rateper entity can be used, however one or more embodiments of the presentinvention can also use more advanced methods which are available or maybecome available, all of which are incorporated herein by reference. Theaffinity matrix is then updated in step 1311 and the nested loops arerespectively looped back to process the next iteration. Finally, theAffinity Matrix is returned to the calling function in step 1313. Itshould be further noted that for example purposes no optimization ofprocessing efficiency was depicted and that embodiments of the presentinvention may further optimize this function. A person skilled in theart would appreciate that such optimizations do not alter the corefunctionality of this function.

Table 21 below illustrates an exemplary pseudo code implementation of aprogram function used to create an “affinity matrix” between users andentities, as applicable to one or more embodiments of the presentinvention.

TABLE 21 Line (FIG./ Step) Comment Pseudo Code 1600 Creates an “affinityGenerate_Users_Affinity_Matrix(State_Transition_Matrix, (13/ matrix”between users and Observations Matrix, User_predictions) 1315) entities.The goal is to determine which entities are liked by each user and whicharen't 1610 Initialize the object that If (Affinity_Matrix.sizeof( ) =0) then Affinity_Matrix<- (13/ will be returned (new) 1305) 1620 Iterateevent elements Foreach (t in Split_To_Time_Intervals(event.length)) (13/1306) 1630 Iterate states  Foreach (State in State_Transition_Matrix)(13/ 1307) 1640 Iterate observations   Foreach (Observation inObservation_States) (13/ 1308) 1650 Iterate users    Foreach (User inUser_Prediction.Get_Users( )) (13/ 1309) 1660 Taking into account theAffinity<-Determine_Observation_Affinity(State, (13/ time, state,observation Observeation, User_Predictions[User][t], 1310) and the inputprovided by event.State_At(t)) the user (among others and withoutlimitation), determine whether the prediction was (un)favorable enoughto deduce there's an affinity. 1670 Update the final objectUpdate_Affinity_Matrix(Affinity_Matrix, affinity) (13/ with the user'saffinities 1311) 1680 End for (13/ 1312) 1690 End For (13/ 1312) 1695End For (13/ 1312) 1696 End For (13/ 1312) 1697 An example of the ReturnAffinity_Matrix (13/ returned affinity matrix is 1313) presented inTable 12.

FIG. 14 represents an exemplary computer function for the main programexecution loop of an embodiment of the present invention. Processinggroup 1414 includes input data elements including event data, initialtime interval, and the State Transition and Observation Matrices. Mainprocessing group 1415 includes the initial event state query step 1404,passing control to the main processing loop 1405 which preferably runsfor the selected duration (and optionally longer or shorter) of theselected event. Step 1406 calls the function depicted in FIG. 8 , andthe user predictions are then normalized to one in step 1407. Step 1408calls the function depicted in FIG. 15 , to generate observations forthe current event interval, which are then passed to the Hidden MarkovModel 1409 to calculate the most probable sequence of occurrences. Useraffinity is then determined by calling the function in FIG. 13 , whichupdates the User Affinity Matrix that is then optionally processed bystep 1411 by calling the function depicted in FIG. 16 . A log of datathe current state of the event, user actions and predictions can berecorded in step 1416. The current state of the event is updated in step1412 before looping back in step 1413 to process the next occurrencewithin the event.

Table 22 below illustrates an exemplary pseudo code implementation of amain program function used to create the main program execution loop forone or more embodiments of the present invention.

TABLE 22 Line (FIG./ Step) Comment Pseudo Code 2000 Main programfunction Main(event, time_interval, State_Transition_Matrix, (14/ forsystem runtime Observation_Matrix) 1415) operations. 2005 Initialize thecurrent state Current_State<-event.Get_Initial_State( ) (14/ variable.1404) 2010 Starts a loop that iterates Foreach (t inSplit_To_Time_Intervals(event.length, (14/ through each occurrenceCurrent_State)) 1405) within a series of occurrences that include anevent. Actual splits can be time based, rule based, play based, externalinput (human, data or otherwise) or otherwise, and any combinationthereof. 2020 For each occurrence of the  User_predictions_1[t] <- (14/selected event, call Generate_User_Predictions_For_Time_Interval(t-1,1406) function 1100 (FIG. 8) to event, False) read user predicationsthat were previously captured (prior to the selected occurrence of thesame event or using data from past events, or any combination thereof)for each of the questions about said selected occurrence. 2022 For eachoccurrence of the  User_predictions_2[t]<- (14/ selected event, functionGenerate_User_Predictions_For_Time_Interval(t+1, 1406) 1100 (FIG. 8) iscalled to event, True) read user predications that were captured aftersaid selected occurrence (following the selected occurrence of the sameevent or using data from other events, or any combination thereof) foreach of the questions about said selected occurrence. 2024 Next, the twotypes of  User_predictions[t]<- (14/ predictions (past andCombine_Predictions(User_predictions_1, 1407) future) are combined intoUser_Predicitons_2) one matrix . . . 2030 . . . and normalized.User_predictions[t]<- (14/ Normalize_Predictions(User_predictions[t])1407) 2040 Call Function 2100 on Observation_States<- (14/ FIG. 15 tocalculate what Generate_Observation_States(event, User_predictions)1408) are the likely states based on the predictions of the users. 2050In at least one States_Sequence<- (14/ embodiment, use theCalculate_Likely_States_Sequence(Observation_States, 1409) backwardsrecursion Observations_Matrix, State_Transition_Matrix) algorithm forHMM. 2060 Get the user's current Affinity<- (14/ affinity based on thelatest Determine_Observation_Affinity(States_Se- 1410) prediction.quence.Get_Current_State( ), Observation, User_Predictions[User][t],event.State At(t)) 2065 Record event log, eventLog<-(States_Sequence.Get_Current_State( ), (14/ timeline, user actions.Observation, User_Predictions[User][t], 1416) event.State_At(t)) 2070Monetization and Generate_And_Send_Relevant_Content(event, (14/commercial integration State_Sequence, affinity, Affinity_Matrix) 1411)options are called via function 2200 on FIG. 16, (using data from boththe present affinity calculation and the Affinity Matrix from thetraining phase). 2075 Now, update the current Current_State<- (14/ stateand assess several Determine_Current_State(User_predictions[t]) 1412)likely options, choosing the smallest time interval that fits one ofthem. This way it is less likely to miss anything, while also notcreating spurious questions that are sent to users. 2080 End For (14/1413)

FIG. 15 represents a computer function to create or update observationsbased on the selected events and the user predictions matrix that wascreated during the training phase. Processing group 1507 includes inputdata elements including event data and the User Predictions Matrix.Processing group 1508 includes initialization step 1503 which convertsprevious user predictions into initial potential observations, followedby step 1504 which calculates the probability of all observationsprocessed by the previous step based on analysis of previous (ortrained) user predictions, using, in one or more embodiments, UserPredictions Matrix 903. All event state transitions that are over aprobability threshold are then created in step 1505, with the processeddata returned to the calling function in step 1506.

Table 23 below illustrates an exemplary pseudo code implementation of aprogram function used to generate a list of observations based on theselected events and the user predictions matrix, as applicable to one ormore embodiments of the present invention.

TABLE 23 Line (FIG./ Step) Comment Pseudo Code 2100 Program function toGenerate_Observation_States(event, (15/ generate list ofUser_predictions) 1508) observations based on the selected events andthe user predictions matrix that was created during the training phase.2110 Populate initial Initial Observations<- (15/ observations matrix.Translate_Predicitons_To_Observations(event, 1503) User_predictions)2120 Calculate probability of Likely_Observations<- (15/ observations.Obtain_Top_Likely_Observations(event, 1504) User_predictions) 2130Generate the most “likely” Chosen_States<- (15/ states based on thechosen Generate_Chosen_States_And_Probabilities(event, 1505) likelyobservations. These Likely_Observations) states are used in the HMMmodel in order to deduce the most likely sequence of states. 2140 Returnmatrix of chosen Return Chosen_States (15/ states to the calling 1506)function.

FIG. 16 represents a computer function to create and transmituser-directed content as a result of a user's affinity to a particularentity in the event. This function may be used to call an externaladvertising system, for example, signaling the user's information andenabling display of relevant advertising, promotional messaging, orother similar content on a user's device as a result of the user'sactions, their affinity, and possibly in relation to any entity.Processing group 1611 includes input data elements including event data,the desired affinity relationship and the State Sequence and AffinityMatrices. Processing group 1612 contains two nested loops, iteratingusers 1604 and active entities 1605 respectively. For each combination,the relevance 1603 of the affinity to the iterated entity is evaluatedfirst, followed by evaluation 1606 of the affinity of the specific useris evaluated against all entities, with any affinity value over athreshold is sent to step 1607, which generates relevant content orsignals and transmits those to the content publication step, 1608, anexample of which is depicted in step 2004 of FIG. 20 . In either case(above or below affinity threshold) control is then returned to step1609 and the nested iteration loops process their next respectiveiteration. Finally, the updated Affinity Matrix is returned to thecalling function in step 1610.

Table 24 below illustrates an exemplary pseudo code implementation of aprogram function used to create and transmit user-directed content as aresult of a user's affinity to a particular entity in the event, asapplicable to one or more embodiments of the present invention.

TABLE 24 Line (FIG./ Step) Comment Pseudo Code 2200Generate_And_Send_Relevant_Content (event, (16/ State_Sequence,Affinity, Affinity_Matrix) 1612) 2210 Iterate Active Users for Foreach(User in User_Prediction.Get_Users( )) (16/ Event 1604) 2220 IterateActive Entities for  Foreach (Entity in event.Entities) (16/ Event 1605)2230 Evaluate affinity   If (State_Sequence.Entity_Is_Relevant(Entity)(16/ relevance against iterated 1603) entity 2240 Evaluate affinity   If (Positive_Affinity_Exists(User, event, (16/ connection againstiterated Affinity, Affinity_Matrix) 1606) entity 2250 Publishadvertising or Generate_And_Publish_Content_To_User(User, (16/ othercontent to user Entity, event.Get_Event_Advertisment( )) 1607) device,for example as illustrated on step 2004 of FIG. 20. 2285 End For (16/1609) 2285 Return Affinity Matrix to Return (Affinity_Matrix) (16/calling function. 1610)

FIG. 17 represents a computer function to analyze user responses anddetermine whether a sufficient number of users are available with asufficient legitimacy score. Processing group 1711 includes input to thefunction, including response time data, the level of event activity, andthe user details matrix. Within main execution group 1712, Step 1704initializes the legitimate users matrix, followed by iterative loop 1705which iterates all users, passing them to step 1706 which thendetermines if each If the user's reliability score is above or below thethreshold returned by the Minimum_Legitimacy_Threshold( ) function.Users with scores below the threshold are skipped, and users with scoresabove the threshold are passed on to step 1707 which compares theresponse data (including, but not limited to, response time, connectiondata, latency data, and so on) of the user is consistent with that of amajority of the users whose geo-location is similar to that user. Asbefore, users with inconsistencies are skipped and users with data thatis consistent are passed to step 1708 which marks that user aslegitimate in the Legitimate_User_Details table. Finally, step 1710returns the estimated response reliability of theLegitimate_User_Details table to the calling function.

Table 25 below illustrates an exemplary pseudo code implementation of aprogram function used to analyze user responses and to determine whetherthere is a sufficient number of users are available with a sufficientlegitimacy score, as applicable to one or more embodiments of thepresent invention.

TABLE 25 Line (FIG./ Step) Comment Pseudo Code 1700 Analyzes the answersCalculate_User_Response_Time_ (17/ provided by all the usersFraud_Likelyhood (Response_Time, 1712) and determines whetherEvent_Dynamic_Level, User_Details) there is a sufficient number of userswith a sufficient legitimacy score. 1710 Initialize the object thatLegitimate_User_Details<-(new) (17/ will contain the users 1704) whosereliability exceeds a predefined threshold. 1720 Iterate users Foreach(User in User_Details) (17/ 1705) 1730 If the user's reliability is If(user.Legitimacy_Score < (17/ above the threshold, it willMinimum_Legitimacy_Threshold( ) 1706) be used for analysis later 1740Continue (17/ 1706) 1750 If the response of the user If(User_Response_Is_Consistent_ (17/ is consistent with that of aWith_Proximate_Users(User, User_Details)) 1707) majority of the userswhose geo-location is similar to his, then . . . 1760 . . . add the userto the list Legitimate_User_Details.Add(User) (17/ of legitimate users.1708) 1770 End For (17/ 1709) 1780 Return estimated number Return(Estimate_Response_ (17/ and reliability of theReliability(Legitimate_User_Details)) 1710) remaining users anddetermine a legitimacy score (the higher the better).

Computing System

Referring now to FIG. 18 , there is illustrated a block diagram of acomputer operable to execute the disclosed architecture. In order toprovide additional context for various aspects of the subject invention,FIG. 18 and the following discussion are intended to provide a brief,general description of a suitable computing environment in which thevarious aspects of the invention can be implemented. While the inventionhas been described above in the general context of computer-executableinstructions that may run on one or more computers, those skilled in theart will recognize that the invention also can be implemented incombination with other program modules and/or as a combination ofhardware and software.

Generally, program modules include routines, programs, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. Moreover, those skilled in the art will appreciatethat the inventive methods can be practiced with other computer systemconfigurations, including single-processor or multiprocessor computersystems, minicomputers, mainframe computers, as well as personalcomputers, handheld computing devices, microprocessor-based orprogrammable consumer electronics, and the like, each of which can beoperatively coupled to one or more associated devices, including variousarchitectures such as cloud computing.

The illustrated aspects of the invention may also be practiced indistributed computing environments where certain tasks are performed byremote processing devices that are linked through a communicationsnetwork. In a distributed computing environment, program modules can belocated in both local and remote memory storage devices.

A computer typically includes a variety of computer-readable media.Computer-readable media can be any available media that can be accessedby the computer and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media can include computer storage mediaand communication media. Computer storage media includes both volatileand nonvolatile, removable and non-removable media implemented in anymethod or technology for storage of information such as computerreadable instructions, data structures, program modules or other data.Computer storage media includes, but is not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digital videodisk (DVD) or other optical disk storage, magnetic cassettes, magnetictape, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store the desired information andwhich can be accessed by the computer.

Communication media typically embodies computer-readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism, and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, cellular, infrared and other wireless media. Combinationsof the any of the above should also be included within the scope ofcomputer-readable media.

With reference again to FIG. 18 , there is illustrated an exemplaryenvironment 1801 for implementing various aspects of the invention thatincludes a computer 1802, the computer 1802 may include a processingunit 1803, a system memory 1804 and a system bus 1805.

The computer 1802, may, in embodiments, correspond to any suitable typeof electronic device including, but are not limited to, desktopcomputers, mobile computers (e.g., laptops, ultrabooks), mobile phones,portable computing devices, such as smart phones, tablets and phablets,televisions, set top boxes, smart televisions, personal display devices,large scale display devices (e.g., billboards, street signs, etc.),personal digital assistants (“PDAs”), gaming consoles and/or devices,virtual reality devices, smart furniture, smart household devices (e.g.,refrigerators, microwaves, etc.), smart vehicles (e.g., cars, trucks,motorcycles, etc.), smart transportation devices (e.g., boats, ships,trains, airplanes, etc.), wearable devices (e.g., watches,pins/broaches, headphones, etc.), smart security systems, and/or smartaccessories (e.g., light bulbs, light switches, electrical switches,etc.), to name a few. In some embodiments, the computer 1802 may berelatively simple or basic in structure such that no, or a minimalnumber of, mechanical input option(s) (e.g., keyboard, mouse, track pad)or touch input(s) (e.g., touch screen, buttons) are included. Forexample, the computer 1802 may be able to receive and output audio, andmay include power, processing capabilities, storage/memory capabilities,and communication capabilities. However, in other embodiments, thecomputer 1802 may include one or more components for receivingmechanical inputs or touch inputs, such as a touch screen and/or one ormore buttons.

The computer 1802 may, in embodiments, be a voice activated electronicdevice. A voice activated electronic device, as described herein, maycorrespond to any device capable of being activated in response todetection of a specific word (e.g., a word, a phoneme, a phrase orgrouping of words, or any other type of sound, or any series oftemporally related sounds). For example, a voice activated electronicdevice may be one or more of the following: Amazon Echo®; Amazon EchoShow®; Amazon Echo Dot®; Smart Television (e.g., Samsung® Smart TVs);Google Home®; Voice Controlled Thermostats (e.g., Nest®; Honeywell®Wi-Fi Smart Thermostat with Voice Control), smart vehicles, smarttransportation devices, wearable devices (e.g., Fitbit®), and/or smartaccessories, to name a few.

The computer 1802 can further include an internal hard disk drive (HDD)1808A (e.g., EIDE, SATA, NVMe), which internal hard disk drive 1808A mayalso be configured for external use in a suitable chassis (not shown), amagnetic floppy disk drive (FDD) 1809, (e.g., to read from or write to aremovable diskette 1810) and an optical disk drive 1811, (e.g., readinga CD-ROM disk 1812 or, to read from or write to other high capacityoptical media such as the DVD). The hard disk drive 1808A, magnetic diskdrive 1809 and optical disk drive 1811 can be connected to the systembus 1805 by a hard disk drive interface 1813, a magnetic disk driveinterface 1814 and an optical drive interface 1815, respectively. Theinterface 1813 for external drive implementations includes at least oneor more of Universal Serial Bus (USB) and IEEE 1394 interface, PCIe,Thunderbolt, SCSI, or SAS technologies. The drives and their associatedcomputer-readable media provide nonvolatile storage of data, datastructures, computer-executable instructions, and so forth. For thecomputer 1802, the drives and media accommodate the storage of any datain a suitable digital format. Although the description ofcomputer-readable media above refers to a HDD, a removable magneticdiskette, and a removable optical media such as a CD or DVD, it shouldbe appreciated by those skilled in the art that other types of mediawhich are readable by a computer, such as zip drives, magneticcassettes, flash memory cards, cartridges, and the like, may also beused in the exemplary operating environment, and further, that any suchmedia may contain computer-executable instructions for performing themethods of the invention. The computer 1802 and the components thereof,described more fully below, may be applicable to remote computer(s)1825, the description of which applying.

Processing unit 1803 may include any suitable processing circuitrycapable of controlling operations and functionality of the computer1802, as well as facilitating communications between various componentswithin the computer 1802. In some embodiments, processing unit 1803 mayinclude a central processing unit (“CPU”), a graphic processing unit(“GPU”), one or more microprocessors, a digital signal processor, or anyother type of processor, or any combination thereof. In someembodiments, the functionality of processing unit 1803 may be performedby one or more hardware logic components including, but not limited to,field-programmable gate arrays (“FPGA”), application specific integratedcircuits (“ASICs”), application-specific standard products (“ASSPs”),system-on-chip systems (“SOCs”), and/or complex programmable logicdevices (“CPLDs”). Furthermore, each of processing unit 1803 may includeits own local memory, which may store program systems, program data,and/or one or more operating systems. However, processing unit 1803 mayrun an operating system (“OS”) for the computer 1802, and/or one or morefirmware applications, media applications, and/or applications residentthereon. In some embodiments, the processing unit 1803 may run a localclient script for reading and rendering content received from one ormore websites. For example, the processing unit 1803 may run a localJavaScript client for rendering HTML or XHTML content received from aparticular URL accessed by the computer 1802. The processing unit 1803may include one or more processor(s).

The system bus 1805 may operationally couple the following system 1801components, which may include, but not limited to, the system memory1804 to the processing unit 1803. The processing unit 1803 can be any ofvarious commercially available processors. Dual microprocessors andother multi-processor architectures may also be employed as theprocessing unit 1803. The system bus 1805 can be any of several types ofbus structure that may further interconnect to a memory bus (with amemory controller), a memory bus (without a memory controller), aperipheral bus, and a local bus using any of a variety of commerciallyavailable bus architectures. The system memory 1804 may include readonly memory (ROM) 1806 and random access memory (RAM) 1807. A basicinput/output system (BIOS) is stored in a non-volatile memory 1806 suchas ROM, EPROM, EEPROM, which BIOS contains the basic routines that helpto transfer information between elements within the computer 1802, suchas during start-up. The RAM 1807 can also include a high-speed RAM suchas static RAM for caching data.

The computer 1802 may include system memory 1807, internal storage1808A, and/or external storage 1808B. System memory 1807, internalstorage 1808A, and/or external storage 1808B may include one or moretypes of storage mediums such as any volatile or non-volatile memory, orany removable or non-removable memory implemented in any suitable mannerto store data for computer 1802. For example, information may be storedusing computer-readable instructions, data structures, and/or programsystems. Various types of storage/memory may include, but are notlimited to, hard drives, solid state drives, flash memory, permanentmemory (e.g., ROM), electronically erasable programmable read-onlymemory (“EEPROM”), CD-ROM, digital versatile disk (“DVD”) or otheroptical storage medium, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, RAID storage systems, or anyother storage type, or any combination thereof. Furthermore, systemmemory 1807 may be implemented as computer-readable storage media(“CRSM”), which may be any available physical media accessible byprocessing unit 1803 to execute one or more instructions stored withinsystem memory 1807, internal storage 1808A, and/or external storage1808B. In some embodiments, one or more applications (e.g., mobileapplication software 3000, gaming, music, video, calendars, lists,banking, social media etc.) may be run by processing unit 1803, and maybe stored in system memory 1807, internal storage 1808A, and/or externalstorage 1808B.

Continuing to refer to FIG. 18 , a number of program modules can bestored in the drives and RAM 1807, including an operating system 1816,one or more application programs 1817, other program modules 1818 andprogram data 1819. All or portions of the operating system,applications, modules, and/or data can also be cached in the RAM 1807.It is appreciated that the invention can be implemented with variouscommercially available operating systems or combinations of operatingsystems. A user can enter commands and information into the computer1802 through one or more wired/wireless input devices, e.g., a keyboard1820 and a pointing device, such as a mouse 1821. Other input devices(not shown) may include a microphone, an IR remote control, a joystick,a game pad, a stylus pen, touch screen, or the like. These and otherinput devices are often connected to the processing unit 1803 through aninput device interface 1822 that is coupled to the system bus 1805, butcan be connected by other interfaces, such as a parallel port, an IEEE1394 serial port, a game port, a USB port, an IR interface, etc.

FIG. 18 further illustrates a monitor 1823 or other type of displaydevice can be connected to the system bus 1805 via an interface, such asa video adapter 1824. In addition to the monitor 1823, a computertypically includes other peripheral output devices (not shown), such asspeakers, printers, etc., without limitation. The monitor 1823, maycorrespond to a display device and/or touch screen, which may be anysize and/or shape and may be located at any portion of the computer1802. Moreover, the monitor 1823 may be operationally connected to thecomputer 1802 (e.g. connected via one or more cables and/or wires,wireless connection, etc., to name a few). Various types of displaydevices may include, but are not limited to, liquid crystal displays(“LCD”), LED, OLED, QLED, monochrome displays, color graphics adapter(“CGA”) displays, enhanced graphics adapter (“EGA”) displays, videographics array (“VGA”) display, or any other type of display, or anyvariation or combination thereof. Still further, a touch screen may, insome embodiments, correspond to a display device including capacitivesensing panels capable of recognizing touch inputs thereon. Forinstance, the monitor 1823 may correspond to a projected capacitivetouch (“PCT”), screen include one or more row traces and/or driving linetraces, as well as one or more column traces and/or sensing lines. Insome embodiments, the monitor 1823 may be an optional component for thecomputer 1802. For instance, the computer 1802 may not include themonitor 1823. Such devices, sometimes referred to as “headless” devices,may output audio, or may be in communication with a display device foroutputting viewable content.

The monitor 1823, in one non-limiting embodiment, may include aninsulator portion, such as glass, coated with a transparent conductor,such as indium tin oxide (“InSnO” or “ITO”). In general, one side of thetouch screen display may be coated with a conductive material. A voltagemay be applied to the conductive material portion generating a uniformelectric field. When a conductive object, such as a human finger,stylus, or any other conductive medium, contacts the non-conductiveside, typically an outer surface of the monitor 1823, a capacitancebetween the object and the conductive material may be formed. Theprocessing unit 1803 may be capable of determining a location of thetouch screen associated with where the capacitance change is detectedand may register a touch input as occurring at that location.

In some embodiments, the monitor 1823 may include multiple layers, suchas a top coating layer, a driving line layer, a sensing layer, and aglass substrate layer. As mentioned previously, the glass substratelayer may correspond to an insulator portion, while the top coatinglayer may be coated with one or more conductive materials. The drivingline layer may include a number of driving lines, and the sensing layermay include a number of sensing lines, which are described in greaterdetail below. One or more additional layers, or spaces between layers,may be included. Furthermore, any suitable number of driving lines andsensing lines for driving the line layer and the sensing layer,respectively, may be used.

In some embodiments, the driving lines and the sensing lines of thedriving line layer and the sensing line layer, respectively, may form anumber of intersection points, where each intersection functions as itsown capacitor. Each sensing line may be coupled to a source, such that acharge is provided to each sensing line, and changes in capacitance of aparticular driving line and sensing line are detectable thereby. Inresponse to a conductive object being brought proximate, orsubstantially touching an outer surface of the top coating layer, amutual capacitance of a particular capacitor (e.g., an intersectionpoint) may reduce in magnitude. In other words, a voltage drop may bedetected at a location on the monitor 1823 corresponding to where aconductive object contacted the monitor 1823.

A change in capacitance may be measured to determine a location on thetouch screen where the object has contacted the surface. For example, ifan individual touches a point on the monitor 1823, then a correspondingdriving line and sensing line that intersect at that point may beidentified. A location of the point may have one or more pixelsassociated with that location, and therefore one or more actions may beregistered for an item or items that are displayed at that location. Theprocessing unit 1803 of the computer 1802 may be configured to determinewhich pixels are associated with a particular location point, and whichitem or items are also displayed at that pixel location. Furthermore,the computer 1802 may be configured to cause one or more additionalactions to occur to the item or items being displayed on the monitor1823 based on a temporal duration the touch input, and or if one or moreadditional touch inputs are detected. For example, an object (e.g. auser's hand, a stylus, etc., to name a few) that is contacted on themonitor 1823 at a first location may be determined, at a later point intime, to contact the monitor 1823 at a second location. In theillustrative example, the object may have initially contacted themonitor 1823 at the first location and moved along a particular drivingline to the second location. In this scenario, a same driving line mayhave detected a change in capacitance between the two locations,corresponding to two separate sensing lines.

The number of driving lines and sensing lines, and therefore the numberof intersection points, may directly correlate to a “resolution” of atouch screen. For instance, the greater the number of intersectionpoints (e.g., a greater number of driving lines and sensing lines), thegreater precision of the touch input. For instance, a touch screenmonitor 1823 having 100 driving lines and 100 sensing lines may have 100intersection points, and therefore 100 individual capacitors, while atouch screen monitor 1823 having 10 driving lines and 10 sensing linesmay only have 10 intersection points, and therefore 10 individualcapacitors. Therefore, a resolution of the touch screen having 100intersection points may be greater than a resolution of the touch screenhaving 10 intersection points. In other words, the touch screen having100 intersection points may be able to resolve a location of an objecttouching the touch screen with greater precision than the touch screenhaving 10 intersection points. However, because the driving lines andsensing lines require a voltage to be applied to them, this may alsomean that there is a larger amount of power drawn by the computer 1802,and therefore the fewer driving lines and/or sensing lines used, thesmaller the amount of power that is needed to operate the touch screendisplay.

In some embodiments, the monitor 1823 may correspond to ahigh-definition (“HD”) display. For example, the monitor 1823 maydisplay images and/or videos of 720p, 1080p, 1080i, or any other imageresolution. In these exemplary scenarios, the monitor 1823 may include apixel array configured to display images of one or more resolutions. Forinstance, a 720p display may present a 1024 by 768, 1280 by 720, or 1366by 768 image having 786,432; 921,600; or 1,049,088 pixels, respectively.Furthermore, a 1080p or 1080i display may present a 1920 pixel by 1080pixel image having 2,073,600 pixels. However, the aforementioned displayratios and pixel numbers are merely exemplary, and any suitable displayresolution or pixel number may be employed for the monitor 1823, such asnon-HD displays, 4K displays, and/or ultra displays.

The computer 1802 may operate in a networked environment using logicalconnections via wired and/or wireless communications to one or moreremote computers, such as a remote computer(s) 1825. The remotecomputer(s) 1825 may be a workstation, a server computer, a router, apersonal computer, portable computer, microprocessor-based entertainmentappliance, a peer device or other common network node, to name a few.The remote computer(s) 1825 may include many or all of the elementsdescribed relative to the computer 1802, although, for purposes ofbrevity, only a memory storage device 1826 is illustrated.

The logical connections depicted include wired/wireless connectivity toa local area network (LAN) 1827 and/or larger networks, e.g., a widearea network (WAN) 1828. Such LAN and WAN networking environments arecommonplace in offices, and companies, and facilitate enterprise-widecomputer networks, such as intranets, all of which may connect to aglobal communication network, e.g., the Internet. For the purposes ofthe present invention, any and all handheld wireless devices such asApple Inc.'s iPhone, iPad, any device running Google's Android operatingsystem, or similar devices made by BlackBerry and others are alsoconsidered computers. When used in a LAN networking environment, thecomputer 1802 is connected to the local network 1827 through a wiredand/or wireless communication network interface or adapter 1829. Theadaptor 1829 may facilitate wired or wireless communication to the LAN1827, which may also include a wireless access point disposed thereonfor communicating with the wireless adaptor 1829.

The computer 1802 and/or remote computer(s) 1825 may, in embodiments,use any communications protocol, such as any of the previously mentionedexemplary communications protocols. In some embodiments, computer 1802and/or remote computer(s) 1825 may include one or more antennas tofacilitate wireless communications with a network using various wirelesstechnologies (e.g., Wi-Fi, Bluetooth®, radiofrequency, etc.). In yetanother embodiment, computer 1802 and/or remote computer(s) 1825 mayinclude one or more universal serial bus (“USB”) ports, one or moreEthernet or broadband ports, and/or any other type of hardwire accessport so that computer 1802 and/or remote computer(s) 1825 are able tocommunicate with one another or with one or more communicationsnetworks.

The computer 1802 depicted in FIG. 18 can be operable to communicatewith any wireless devices or entities operatively disposed in wirelesscommunication, e.g., a printer, scanner, desktop and/or portablecomputer, portable data assistant, communications satellite, any pieceof equipment or location associated with a wirelessly detectable tag(e.g., a kiosk, news stand, restroom), and telephone. This includes atleast Wi-Fi and Bluetooth® wireless technologies. Thus, thecommunication can be a predefined structure as with a conventionalnetwork or simply an ad hoc communication between at least two devices.When used in a WAN networking environment, the computer 1802 can includea modem 1830, or is connected to a communications server on the WAN1828, or has other means for establishing communications over the WAN1828, such as by way of the Internet. The modem 1830, which can beinternal or external and a wired or wireless device, is connected to thesystem bus 1805 via the serial port interface 1822. In a networkedenvironment, program modules depicted relative to the computer 1802, orportions thereof, can be stored in the remote memory/storage device1826. It will be appreciated that the network connections shown areexemplary and other means of establishing a communications link betweenthe computers can be used.

Referring now to FIG. 19 , a networked system according to one or moreembodiments can include a system server 1901 having application 1902 andstorage 1903 intercommunicably connected to each other and to a varietyof client devices via a plurality of wired and wireless connections.System Server 1901 is running at least one component of one or moreembodiments of the present invention 1902 and server 1901 may includefurther operating, memory and storage features as characterized for acomputing system in FIG. 18 . System Server 1901 may be similar to thecomputer 1802, described above in connection with FIG. 18 , thedescription of which applying herein. Storage 1903 may be similar tosystem memory 1804, internal storage 1808A, and/or external storage1808B described above in connection with FIG. 18 , the descriptions ofwhich applying herein. Connection to network 1910 (which, for example,without limitation, can include the Internet, in full or in part) can behad via any combination of wired or wireless connections 1916.

System server 1901, user device 1909, and/or devices 1909, 1907 and 1904(devices 1907 and 1904 being grouped as a first group of users 1920using application 1905) may communicate (e.g. transfer data, receivedata, etc.) over the network 1910, such as the Internet. For example,the network 1910 (which may be one or more networks) may be accessedusing Transfer Control Protocol and Internet Protocol (“TCP/IP”) (e.g.,any of the protocols used in each of the TCP/IP layers), HypertextTransfer Protocol (“HTTP”), WebRTC, SIP, and wireless applicationprotocol (“WAP”), are some of the various types of protocols that may beused to facilitate communications between System server 1901, userdevice 1909, and/or devices 1909, 1907 and 1904. In some embodiments,System server 1901, user device 1909, and/or devices 1909, 1907 and 1904may communicate with one another via a web browser using HTTP. Variousadditional communication protocols may be used to facilitatecommunications between System server 1901, user device 1909, and/ordevices 1909, 1907 and 1904, including, but not limited to, Wi-Fi (e.g.,802.11 protocol), Bluetooth®, radio frequency systems (e.g., 900 MHz,1.4 GHz, and 5.6 GHz communication systems), cellular networks (e.g.,GSM, AMPS, GPRS, CDMA, EV-DO, EDGE, 3GSM, DECT, IS136/TDMA, iDen, LTE orany other suitable cellular network protocol), infrared, BitTorrent,FTP, RTP, RTSP, SSH, and/or VOIP.

User device 1909 can be connected to network 1910 via any combination ofwired or wireless connections 1915, which in turn connects to the systemserver 1901 via connection 1916. User device 1909 may be similar to thecomputer 1802, described above in connection with FIG. 18 , thedescription of which applying herein. A network 1910 can further includea wired Internet network 1915 and/or a wireless network 1914 throughwhich transmissions are distributed by wireless transmitters 1908, aswell as other forms of computing and transmission networks, all withoutlimitation. A wireless device 1907, according to some embodiments, ispreferably a cellular phone, tablet, or other mobile device capable ofconnecting to a network via any combination of wired, wireless,infrared, auditory or similar means. In other embodiments a PDA, acomputer, a laptop 1904 or any other device capable of communicatingwith the server 1901 is possible. An application 1905 stored on thedevice 1907 and/or device 1904, which application can include a webbrowser application and/or a proprietary application, or a combinationof such applications working integrated fashion, the applicationincluding one or more configurations of machine instructions forcontrolling a computer processor. In some embodiments, software toidentify the physical location of the device 1907 or device 1904 can bestored on the devices, and location data as well as other data can bestored in storage media 1913 resident on such devices or stored indistributed storage locations on the network. Storage media 1913 may besimilar to system memory 1804, internal storage 1808A, and/or externalstorage 1808B described above in connection with FIG. 18 , thedescriptions of which applying herein. Furthermore, the devices 1909,1907 and 1904 can receive data that are used to enable the user torespond to questions, and to capture the user's answers as well as thetime in which the user responded. Additional applications are able to beincluded on the server 1901 and on the devices 1909, 1907 and/or 1904,as necessary, for smooth operation of the process and method, accordingto one or more embodiments. Although some of the applications aredescribed separately above, in some embodiments the applications areincluded in one large application, or are accessed via any combinationof applications on the device.

Still referring to FIG. 19 , wireless transceiver 1908 may be WiFi,WiMax, cellular, or other wireless medium (or any other communicationprotocols mentioned above), and may be connected to network 1910 via anycombination of wired or wireless connections (e.g. any communicationprotocols mentioned above). User devices 1907 and 1904 can be connectedto network 1910 via any combination of wired or wireless connections1914 and to wireless transceiver 1908 running either a standard webbrowser or a customized application by which they access application1902 on server 1901 via any combination of wired or wirelesstransmissions (as mentioned above, devices 1904 and 1907 may communicatevia any of the communications protocol mentioned above). Devices 1904and 1907 may be similar to the computer 1802, described above inconnection with FIG. 18 , the description of which applying herein.

Wi-Fi, or Wireless Fidelity, allows connection to the Internet from acouch at home, a bed in a hotel room, or a conference room at work,without wires. Wi-Fi is a wireless technology similar to that used in acell phone that enables such devices, e.g., computers, to send andreceive data indoors and out; anywhere within the range of a basestation. Wi-Fi networks use radio technologies called IEEE 802.11(a, b,g, etc.) to provide secure, reliable, fast wireless connectivity. AWi-Fi network can be used to connect computers to each other, to theInternet, and to wired networks (which use IEEE 802.3 or Ethernet).Wi-Fi networks operate in the unlicensed 2.4 and 5 GHz radio bands, atan 11 Mbps (802.11a) or 54 Mbps (802.11b/g) data rate, for example, orwith other wireless standards that allow transmission using multiplebands and/or multiple antennas (dual band/MIMO etc as used by 802.11n,802.11ax and so on), so the networks can provide real-world performancesimilar to (or exceeding) the basic 10/100/1000BaseT wired Ethernetnetworks used in many offices. A further embodiment can even allow fortransmission via a wireless method described in RFC 1149 (Seehttp://tools.ietf.org/pdf/rfc1149.pdf, incorporated herein byreference).

Referring now to FIG. 20 , depicted is an illustrative a presentation ofa screenshot of a graphical user interface on a portable clientcomputing device 2001, displaying questions about a past event elementthat the user may respond to. In one or more embodiments of the presentinvention, the user is informed that the questions relate to a pastevent element, in this case by a highlighted graphical object 2002. Theuser may then indicate their past predictions by indicating any of theoptions presented onscreen within region 2003, at which point the deviceshall transmit the user's response back to the system. Region 2004depicts an illustrative example of a first affinity-relatedadvertisement that was generated by the function depicted in FIG. 16 andsent to a user's portable computing device 2001. Computing device 2001may be similar to the computer 1802 described above in connection withFIG. 18 , the description of which applying herein. In embodiments,computing device 2001 may include a display screen, which, may besimilar to the monitor 1823 described above in connection with FIG. 18 ,the description of which applying herein.

Referring now to FIG. 21 , depicted is an illustrative a presentation ofa screenshot of a graphical user interface on a portable clientcomputing device 2101, displaying questions about a future possibleevent element so that the user may choose to predict what they believewill happen in the future. In one or more embodiments of the presentinvention, the user is informed that the questions relate to a pastevent element, in this case by a highlighted graphical object 2002. Theuser may then indicate their past predictions by indicating any of theoptions presented onscreen within region 2003, at which point the deviceshall transmit the user's response back to the system. Region 2104depicts an illustrative example of a second affinity-relatedadvertisement that was generated by the function depicted in FIG. 16 andsent to a user's portable computing device 2001. Portable clientcomputing device 2201 may be similar to the computer 1802 describedabove in connection with FIG. 18 , the description of which applyingherein. In embodiments, the portable client computing device 2101 mayinclude a display screen, which, may be similar to the monitor 1823described above in connection with FIG. 18 , the description of whichapplying herein.

Example 12: Method of Generating an Accurate News Report

Referring now to FIG. 22 , there are illustrated flow charts of acomputer-implemented process for generating an accurate news reportbased on information provided by one or more users of a plurality ofusers of an electronic computer network (e.g., network 1910 in FIG. 19 )in accordance with an exemplary embodiment of the present invention.

The process of FIG. 22 may begin at step S2202. Referring to FIG. 22 ,at step S2202, identification information associated with each user ofthe plurality of users (e.g. users associated with the plurality ofdevices 3006) of the electronic computer network is received by acomputing device (e.g., computer 1802 in FIG. 18 , server 1901 in FIG.19 , computing device 2702 in FIGS. 27-42 , and/or one or morecomponents thereof, to name a few). In embodiments, the identificationinformation may include one or more of the following: a user accountassociated with a user; an e-mail address associated with a user; a nameassociated with a user; biometric data associated with a user; genderinformation of the user associated with the user device; age of the userassociated with the user device; personal data of a user associated withthe user device, which is either volunteered by the user or received viaaccess that is consented to by the user; past and/or present locationinformation associated with the user device; identification informationrelated to a user device associated with a user of the plurality ofusers (e.g. metadata, device type, etc., to name a few), electronicidentification (e.g. electronic identification card, electronicsignature, etc., to name a few), and/or biometric data of the user (e.g.audio sampling(s) of the user's voice, an image of the user, a video ofthe user, and/or a finger print of the user, to name a few), to name afew.

The computing device 2702 (e.g., a computer system) may determine afirst group of users associated with a first group of user devices (e.g.first group of user devices 2708). For example, if a building at aspecific 1234 5th Street is on fire, the computing device, onceidentification information is received, may generate a list of usersthat are within a predetermined radius of 1234 5th Street. This list ofusers may, in embodiments, be grouped as a first group of users (e.g.first group of user devices 2708 of FIG. 27 ). The predetermined radius,in embodiments, may be relative to the type of event that is occurring.In embodiments, a building that is on fire may be visible by userswithin a half a mile radius of the fire. Thus, continuing the example,the computing device 2702 may determine that the first group of users isevery user that is within a half a mile radius of 1234 5th Street. Inembodiments, the predetermined radius may be much larger for events thatare visible from larger distances. For example, a volcanic eruption maybe visible within a 50-mile radius. In embodiments, the predeterminedradius may be much smaller for events that are only visible from smallerdistances. For example, a delay in a train schedule may only be visiblefrom people within a block of a train station where the train delay isoccurring. In embodiments, there may be multiple predetermined radii (orother geofencing/regionalizing approaches e.g. polygonal, vector orbitmaps, to name a few). Continuing the example of a delay in a trainschedule, there may be multiple predetermined radii (or regions), eachpredetermined radius (or region) of the multiple predetermined radii (orregions) may be located a stop on the train line. In embodiments, eachpredetermined radius (or region) of the multiple predetermined radii (orregion) may be used to create different groups of user devices. Forexample, users within a block of Train Station A may be a first group ofuser devices. In embodiments, users within a block of Train Station Bmay be a second group of user devices. Additionally, in embodiments,users within a block of Train Station N may be an N group of userdevices. In embodiments, users may be grouped by one or more of thefollowing factors: previous user responses, previous activity of theuser; and any of the above listed identification information, to name afew.

At step S2204, the identification information is stored in one or moredatabases by the computing device. In embodiments, the one or moredatabases may be: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. The identification information may be stored in accordance with thereceived identification information. For example, the identificationindication may indicate an age of the users. In this example, thecomputing device 2702 may store the identification information by agerange (e.g. 18-25 ages grouped together, 26-40 ages grouped together,etc.). As another example, the identification information may be storedbased on whether a user is part of a specific group of users. Forexample, identification information of the first group of usersassociated with the first group of user devices 2708 may be storedtogether.

At step S2206, a first stimulus message related to an event is generatedby the computing device. Referring to FIG. 27 , in embodiments,computing device 2702 may generate event stimulus message 2704. Eventstimulus message 2704, may, for example, include a message stating “Isthere any information regarding the event happening near you?” Inembodiments, the event stimulus message 2704 may include executablemachine readable instructions that allow for the computing device 2702to determine when one or more user devices of the first group of userdevices 2708 received, opened, and/or began to respond to the eventstimulus message 2704. For example, the event stimulus message 2704 mayinclude Message Disposition Notification(s) (MDN). The event stimulusmessage 2704 may be generated in response to determining that an eventis occurring. For example, as stated above, if there is a report of afire occurring at 1234 5th Street, the computing device 2702 maygenerate event stimulus message 2704 to determine the validity of thereport of the fire occurring. The event stimulus message 2704 may bespecific to the event happening. For example, event stimulus message2704 may include text that states, “we have received a report that afire is happening at 1234 5th street, can you tell us any informationregarding this report?” Additionally, in embodiments, event stimulusmessage 2704 may provide a link to the report and/or an excerpt of thereport. Furthermore, in embodiments, the event stimulus message 2704 mayalso include user specific information. For example, the event stimulusmessage 2704, may state “Bob, we received a report that a fire ishappening near you at 1234 5th street, can you see it? If so, arefiremen already present at the scene?” In embodiments, the eventstimulus message 2704 may include a question regarding information thatthe computing device 2702 has already confirmed. For example, thecomputing device 2702 may have already confirmed that firemen are at thescene. In this example, the event stimulus message 2704 may state “Bob,we received a report that a fire is happening near you at 1234 5thstreet, can you see it? If so, are firemen already present at the scene?Is there any further details you can tell us?” The first question, “arefireman already present at the scene” may be asked to confirm thereliability of the user. The second question, may be to confirm thereliability of the reporting of the fire. In embodiments, the computingdevice 2702 may only review answers from users that provide the correctanswer to the question, “are fireman already present at the scene.”

In embodiments, the computing device 2702 may determine and/or storeevent start information. Event start information, in embodiments, mayrefer to the time at which the event (the event of the event stimulusmessage 2704) was first reported. In embodiments, event startinformation may refer to the time at which the event (the event of theevent stimulus message 2704) began. The time at which an event began mayonly be based on the reporting of the event, which may need to beauthenticated. This authentication process may be similar to the processdescribed in connection with FIG. 22 and/or the process described inconnection with FIG. 23 , the descriptions of which applying herein.

In embodiments, computing device 2702 may be similar to the computer1802 described above in connection with FIG. 18 , the description ofwhich, including all of the described components, applying herein.

Referring back to FIG. 22 , the first stimulus message may includemachine readable instructions to present an inquiry message on the oneor more user devices of the first group of user devices. In embodiments,the inquiry message may be related to a past event element of the event.For example, a past event element may be whether firemen have respondedto a fire. The event, in this example, would be the fire. The eventelement, in this example, is the presence of the firemen. Inembodiments, the inquiry message may be related to a future eventelement. For example, a future event element may be whether a fire islikely to spread to other buildings. The event, in this example, wouldbe the fire. The event element, in this example, is whether the firewill spread. In embodiments the inquiry message may be related to a pastevent element of the event that was modified or did not take place. Forexample, if the computing device has determined that initial eventinformation included incorrect information regarding the presence offiremen, the computing device 2702 may generate a first stimulus messagethat includes questions regarding the incorrect information. If the userresponds correctly, then the user may be determined as a reliablesource. If the user responds incorrectly, then the user may bedetermined as an unreliable source.

At step S2208, the first stimulus message is transmitted by thecomputing device to a first group of user devices (e.g., a device 1904,1907 in FIG. 19 , a portable client computing device 2001 in FIGS. 20and 21 , first group of devices 2708 in FIG. 27 ) associated with afirst group of users of the plurality of users. Referring back to FIG.27 , computing device 2702, may transmit the event stimulus message 2704to the first group of user devices 2708. In embodiments, the eventstimulus message 2704 may be transmitted over network 1910. Inembodiments, the first group of user devices 2708 may include one ormore of: laptop 2708-A, wearable device 2708-B, and/or cell phone2708-C. First group of user devices 2708, in embodiments, may beassociated with a first group of users of the plurality of users (e.g.plurality of users 3006 described in connection with FIG. 30 ). Laptop2708-A, wearable device 2708-B, and/or cell phone 2708-C may be similarto one or more of the following, the descriptions applying herein:device 1904, 1907 described above in connection with FIG. 19 , and/orportable client computing device 2001 described above in connection withFIGS. 20 and 21 .

At step S2210, a first response is received by the computing device fromone or more user devices of the first group of user devices. Inembodiments, the first response (e.g. first response 2802) may includeone or more of the following: (i) user information specific to therespective user associated with the respective user device thattransmits the first response (e.g. user information 2802A); (ii)responsive information related to the event (e.g. event information2802B); (iii) location information associated with a location of therespective user device associated with the respective user (e.g.location information 2802C); (iv) a timestamp (e.g. timestamp 2802D) (v)proximity information; (vi) audio data associated with the event and/orobservations of the user; (vii) image data associated with the eventand/or observations of a user; and/or (viii) video data associated withthe event and/or observations of a user, to name a few.

In embodiments, user information 2802A may include information that mayenable the computing device 2702 to identify the user device and/or userassociated with the first response 2802. In embodiments, userinformation 2802A may include one or more of the following: a useraccount associated with a user; an e-mail address associated with auser; a name associated with a user; biometric data associated with auser; gender information of the user associated with the user device;age of the user associated with the user device; personal data of a userassociated with the user device, which is either volunteered by the useror received via access that is consented to by the user; locationinformation associated with the user device; identification informationrelated to a user device associated with a user of the plurality ofusers (e.g. metadata, device type, etc., to name a few), electronicidentification (e.g. electronic identification card, electronicsignature, etc., to name a few), to name a few. In embodiments, the userinformation 2802A may include connection information. Connectioninformation may refer to the reliability and/or speed of the wirelessand/or wired connection between the user device associated with thefirst response 2802 (e.g. laptop 2708-A) and the computing device 2702.In embodiments, the connection information may be used to providecontext to the timestamp 2802D and/or any time lag computations that maybe used to determine the reliability of the first response 2802. Forexample, if the connection between the computing device 2702 and thelaptop 2708-A is a slow connection, the below described predeterminedtime lag (described below in connection with timestamp 2802D) may behigher (e.g. an expected time lag) and/or the computing device 2702 maynot use computed time lag to determine reliability. Additionally, forexample, if the connection between the computing device 2702 and thelaptop 2708-A is a fast connection, the expected time lag may be low andthe computing device 2702 may rely more heavily on the timestamp 2802Dto determine the reliability of the first response 2802. In embodiments,the computing device 2702 may compare the identification informationreceived in step S2202 with the user information 2802A received with thefirst response 2802 to determine and/or verify the user and/or userdevice associated with the first response 2802. In embodiments, userinformation 2802A may be automatically included with the first response2802, without requiring any additional input from a user associated withthe responding user device (e.g. laptop 2708-A). In embodiments, userinformation 2802A may be inputted by a user associated with respondinguser device, which may be used for verification and/or securitypurposes. For example, the first stimulus message (e.g. event stimulusmessage 2704) may include a security question prompt, which may bedesigned to obtain specific user information 2802A, in order to verifythe user responding. The prompt may appear when a user opens and/oraccesses the first stimulus message. In embodiments, the prompt mayappear when a user opens and/or accesses a mobile application, website,or user interface associated with the computing device 2702.

In embodiments, event information 2802B may include informationresponsive to the stimulus message (e.g. event stimulus message 2704).For example, event stimulus message 2704 may state, as previously noted,“Is there any information regarding the event happening near yourlocation?” The first response 2802 from the laptop 2708-A, may includeevent information 2802B, which may responsive to the “any information”question. Event information 2802B may include information that isreceived via message (e.g. “Yes, the police are arriving at the sceneright now”) and/or information that is received via answers to a prompt(e.g. a prompt that asks “Are the police arriving at the scene”—wherethe prompt has a few preselected answers—“(a) Yes; (b) No; (c) I don'tknow; (d) Maybe, I only hear sirens” and the user selects one or more ofthe options presented).

In embodiments, location information 2802C, may include one or more ofthe types of the following types of data: longitude data (e.g., thelongitude of a geographical coordinate of the one or more of the firstgroup of user devices 2708 (e.g. laptop 2708-A—as shown in FIG. 28 ));latitude data (e.g., the latitude of a geographical coordinate of theone or more of the first group of user devices 2708 (e.g. laptop 2708-A,wearable device 2708-B, cellphone 2708-C—as shown in FIG. 27 ));altitude data (e.g., the altitude of the one or more of the first groupof user devices 2708 (e.g. laptop 2708-A, wearable device 2708-B,cellphone 2708-C—as shown in FIG. 27 ), which may be measured in meters,feet, and miles, to name a few); speed data (e.g., the instantaneousspeed of the one or more of the first group of user devices 2708 (e.g.laptop 2708-A, wearable device 2708-B, cellphone 2708-C—as shown in FIG.27 ), which may be measured in meters per second, feet per second,kilometers per hour, and miles per hour, to name a few); course data(e.g., the direction in which the one or more of the first group of userdevices 2708 (e.g. laptop 2708-A, wearable device 2708-B, cellphone2708-C—as shown in FIG. 27 ) are traveling, which may be measured indegrees and may be relative to due north and which may have a value of−1 when the value is not determined by the sensor); bearing data (e.g.,a user's relative position or movement in degrees); and/or timestampdata (e.g., the time and/or date at which the first response is sent byone or more of the first group of user devices 2708 (e.g. laptop 2708-A,wearable device 2708-B, cellphone 2708-C—as shown in FIG. 27 )) to namea few. Location information 2802C may be provided automatically, withthe consent of one or more users associated with the one or more of thefirst group of user devices 2708 (e.g. laptop 2708-A, wearable device2708-B, cellphone 2708-C—as shown in FIG. 27 ). Additionally, locationinformation 2802C may be provided to the one or more of the first groupof user devices 2708 (e.g. laptop 2708-A, wearable device 2708-B,cellphone 2708-C—as shown in FIG. 27 ) via one or more sensors (e.g. alocation sensor) of the one or more of the first group of user devices2708 (e.g. laptop 2708-A, wearable device 2708-B, cellphone 2708-C—asshown in FIG. 27 )).

In embodiments the first response may further include proximityinformation. Proximity information may include one or more of thefollowing data types: distance data (e.g., the distance of a device inrelation to a point, which may be for example a user's ear and/oranother device which also may be providing data); accuracy data (e.g., arepresentation of the confidence of the proximity information); andtimestamp data (e.g., the time at which data the proximity informationis received), to name a few. In embodiments, the accuracy data may be ona scale from 0 to 1, where 1 is the highest amount of confidence and 0is the lowest amount of confidence. The scale at which the accuracy datamay be computed on may include: 0 to 1; 0 to 10; 0 to 100; 0 to 1,000,etc., to name a few. The proximity information may further includeproximity data that indicates whether a first user device is within acertain distance of one or more user devices of the first group of userdevices. For example, the proximity information may include whetherlaptop 2708-A is within a certain distance of wearable device 2708-Band/or cellphone 2708-C. In embodiments, the proximity data may be usedto determine the authenticity of the first response received by one ormore user devices. For example, if the first response received from thelaptop 2708-A is inconsistent with responses received from the wearabledevice 2708-B and the cellphone 2708-C, the computing device 2702 mayobtain and analyze the proximity information of the laptop 2708-A ascompared to the wearable device 2708-B and the cellphone 2708-C. If thelaptop 2708-A is within a certain distance of the wearable device 2708-Band the cellphone 2708-C, the computing device 2702 may determine thatthe user associated with the laptop 2708-A may be observing and/orexperiencing the same event as one or more users associated with thewearable device 2708-B and the cellphone 2708-C, and, thus, theinformation received in the first response from the laptop 2708-A may bereliable. However, for example, if the laptop 2708-A is not within acertain distance of the wearable device 2708-B and the cellphone 2708-C,the computing device 2702 may determine that the user associated withthe laptop 2708-A may not be observing and/or experiencing the sameevent as one or more users associated with the wearable device 2708-Band the cellphone 2708-C, and, thus, the information received in thefirst response from the laptop 2708-A may be unreliable.

In embodiments, as mentioned above, the first response may includetimestamp 2802D. The timestamp 2802D may refer to the time in which thefirst response was sent by the one or more user devices of the firstgroup of user devices. The timestamp 2802D may be specific to each ofthe one or more user devices. For example, the first response mayinclude multiple responses from multiple user devices of the first groupof user devices. Continuing the example, if a first user associated withthe laptop 2708-A sends a first response at 7:39 AM, the timestampincluded with the first response from the laptop 2708-A may indicate thesend time of 7:39 AM. Continuing the example, if a second userassociated with wearable device 2708-B sends their respective firstresponse (e.g. a second response) at 7:42 AM, the timestamp includedwith the first response from wearable device 2708-B may indicate thesend time of 7:42 AM. In embodiments, the time stamp 2802D may refer tothe time in which the first was response was received by the computingdevice 2702. Continuing the example, if the first user's first responseis received by computing device 2702 at 7:40 AM, the timestamp includedwith the first response may indicate a received time of 7:40 AM.Additionally, continuing the example, if the second user's firstresponse is received by the computing device 2702 at 7:43 AM, the firstresponse may indicate a received time of 7:43 AM. In embodiments, thetime stamp 2802D may refer to the time at which the first stimulusmessage (e.g. the event stimulus message 2704) was sent to the firstgroup of user devices (e.g. first group of user devices 2708).Continuing the example, if the event stimulus message 2704 was sent tothe first group of devices 2708 at 7:35 AM, the timestamp received withthe first responses from the laptop 2708-A and the wearable device2708-B may indicate the sent time of the event stimulus message of 7:35AM. In embodiments, the time stamp 2802D may refer to when one or moreuser devices of the first group of user devices 2708 opened and/oraccessed the event stimulus message 2704. Continuing the example, thefirst user may have opened and/or accessed the event stimulus message2704 at 7:37 AM. Additionally, the second user may have opened and/oraccessed the event stimulus message 2704 at 7:36 AM. Thus, the timestampof the first user's first response may indicate an opened/accessed timeof 7:37 AM and the timestamp of the second user's first response mayindicate an opened/accessed time of 7:36 AM. In embodiments, the timestamp 2802D may refer to when one or more user devices of the firstgroup of user devices 2708 began responding to the event stimulusmessage 2704. Continuing the example, if the first user began to respondto the event stimulus message 2704 at 7:38 AM, the time stamp associatedwith the first user's first response may indicate the beginning of theresponse time at 7:38 AM. Continuing the example, if the second userbegan to respond to the event stimulus message 2704 at 7:37 AM, the timestamp associated with the second user's first response may indicate thebeginning of the response time at 7:37 AM. In embodiments, the timestamp2802D of the first response 2802 may include one or more of theaforementioned “events”—(1) sent time of the first response; (2)received time of the first response; (3) received time of the firststimulus message; (4) opened time of the first stimulus message; (5)accessed time of the first stimulus message; and/or (6) began draftingfirst response time of the first stimulus message, to name a few.

The timestamp 2802D may be used to authenticate one or more firstresponses and/or filter one or more first responses (e.g. one or moreresponses similar to first response 2802) for accuracy. In embodiments,the timestamp 2802D may be compared with the aforementioned event startinformation. The computing device 2702 may compute a time lag betweenthe occurrence of the event (e.g. represented by event startinformation) and the timestamp 2802D. The time lag, in embodiments, maybe used to determine the authenticity of one or more first responses. Inembodiments, the higher the time lag, the less authentic the response is(for the exemplary reasons that if a lot of time has passed since theevent has occurred, the response, even if it is from a user that hasknowledge of the event, may not be as accurate or may be biased based onother opinions from news sources and/or other people that areunreliable). For example, the computing device 2702 may determine thatthe time lag is too long for the first response to be accurate. Inembodiments, the computing device 2702 may have a predetermined time laglimit (e.g. 30 minutes). If the time lag equals to and/or exceeds thepredetermined time lag limit, the computing device 2702 may determinethat the first response is unreliable and thus filter out the response(a more detailed discussion of reliability is located below inconnection with steps S2214-S2222). In embodiments, the lower the timelag, the more authentic the response is (for the exemplary reasons thatif very little time has passed since the event has occurred, theresponse may be more accurate or may have less of a chance to be biasedbased on other opinions from news sources and/or other people that areunreliable). For example, the computing device 2702 may determine thatthe time lag is low and/or within a predetermined time lag limit, andthus is a reliable response and/or has a higher probability of being areliable response (a more detailed discussion of reliability is locatedbelow in connection with steps S2214-S2222). In embodiments, thepredetermined time lag limit may be relative to the type of event. Forexample, the predetermined time lag limit may be shorter for a caraccident event than it would be for a volcanic eruption. This isbecause, for example, a car accident may have a short window for a userto observe the event, whereas a volcanic eruption may be observed forhours. In embodiments, the predetermined time lag limit may vary fromstimulus message to stimulus message, based on, for example, any numberof factors, including the amount of information requested in thestimulus message, the type of stimulus message, and/or the urgency ofinformation requested in the stimulus message, to name a few. Forexample, the event stimulus message 2704 may have a differentpredetermined time lag limit than a second event stimulus message. Inembodiments, the predetermined time lag limit may also vary from user touser, based on, for example, connectivity information, identificationinformation relative to the user (e.g. age, sex, name, etc.), deviceinformation relative to the user device, and/or the amount ofinformation within the response that is sent by the user device to thecomputing device 2702.

In embodiments, audio data may be included with the first response 2802.The audio data, in embodiments, may be audio data representing anaccount of a user's observation of an event (e.g. the user recordingtheir own voice talking about what is happening, or an audio recordingof the event, to name a few). Audio data, in embodiments, may be used toauthenticate an individual, via an audio file of the user's voice. Forexample, if the audio data represents a recording of the user's voice,the computing device 2702 may compare the audio data to previouslystored recordings of the user's voice. If there is a match, thecomputing device 2702 may authenticate the user. If there is no match,the computing device 2702 may, in embodiments, determine that the useris not authenticated, may determine that the first response 2802 needsto be further authenticated, and/or determine that the first response2802 needs to be scrutinized more heavily in the authentication process.In embodiments, first responses 2802 that include audio data may requirea user to input information about the audio data. For example, theinformation about the audio data may include: whether the recording isof the user, what the recording captures, a summary of the recording,and/or consent for the computing device 2702 to use the audio recordingin a generated news report. The audio data, in embodiments, may includeone or more of the following types of audio data: RAW, AU/SND, WAVE,AIFF, MP3, OGG, and/or RAM, to name a few.

In embodiments, image data may be included with first response 2802. Theimage data, in embodiments, may be image data representing a picture ofthe event (e.g. a picture the user took representing the event). Imagedata, in embodiments, may be used to authenticate an individual, via animage file of the user's face. For example, if the image data representsa picture of the user at the event, the computing device 2702 maycompare the image data to previously stored images of the user's face.If there is a match, the computing device 2702 may authenticate theuser. If there is no match, the computing device 2702 may, inembodiments, determine that the user is not authenticated, may determinethat the first response 2802 needs to be further authenticated, and/ordetermine that the first response 2802 needs to be scrutinized moreheavily in the authentication process. In embodiments, first responses2802 that include image data may require a user to input informationabout the image data. For example, the information about the image datamay include: whether the image includes the user, what the imagecaptures, a summary of the image, and/or consent for the computingdevice 2702 to use the image in a generated news report. The image data,in embodiments, may include one or more of the following types of imagedata: TIFF, JPEG, GIF, PNG, and/or RAW, to name a few.

In embodiments, video data may be included with first response 2802. Thevideo data, in embodiments, may be video data representing a video ofthe event (e.g. a video the user captured representing the event, avideo of the user at the event, a video). Video data, in embodiments,may be used to authenticate an individual, via a video file includingaudio of the user and/or an image of the user's face. For example, ifthe video data represents a video where the user's face is shown and/orthe user talks, the computing device 2702 may compare the video data topreviously stored images of the user's face which may be generated basedon stored videos containing the user's face and/or may compare the videodata to previously stored recordings of the user's voice which may begenerated based on stored videos containing the user's voice. If thereis a match, the computing device 2702 may authenticate the user. Ifthere is no match, the computing device 2702 may, in embodiments,determine that the user is not authenticated, may determine that thefirst response 2802 needs to be further authenticated, and/or determinethat the first response 2802 needs to be scrutinized more heavily in theauthentication process. In embodiments, first responses 2802 thatinclude video data may require a user to input information about thevideo data. For example, the information about the video data mayinclude: whether the video includes the user, what the video captures, asummary of the video, and/or consent for the computing device 2702 touse the video in a generated news report. The video data, inembodiments, may include one or more of the following types of videodata: FLV, AVI, Quicktime, MP4, MPG, WMV, MOV, 3GP, Advances StreamingFormat, to name a few.

In embodiments, the computing device 2702 may use information withinfirst response 2802, including event information 2802B and timestamp2802D, to determine reliability and/or filter out unreliable responses.For example, the computing device 2702 may receive the event information2802B of first response 2802 and timestamp 2802D of first response 2802.In this example, the timestamp 2802D may include the began draftingfirst response time of the first stimulus message and the sent time ofthe first response. In embodiments, the computing device 2702 maycompare the began drafting first response time of the first stimulusmessage and the sent time of the first response to calculate a time lag.This time lag may be used to determine the reliability of the firstresponse 2802. In embodiments, this time lag may be viewed in thecontext of the event information 2802B. If, for example, the time lag ishigh, and the amount of information is high, the computing device 2702may determine that the first response 2802 is reliable and/or has ahigher probability of being reliable because the high lag time may bedue to the amount of information input by the user. If, for example, thetime lag is high, and the amount of information is low, the computingdevice 2702 may determine that the first response 2802 is unreliableand/or has a higher probability of being unreliable because the amountof content sent by the user does not reflect the amount of time spentcrafting the first response 2802. If, for example, the time lag is low,and the amount of information is high, the computing device 2702 maydetermine that the first response 2802 is unreliable and/or has a higherprobability of being unreliable because a user may have not been able tosend a response within the amount of time with the amount of informationcontained in the response.

At step S2212 the computing device (e.g. computing device 2702) storesthe first response (e.g. first response 2802) in the one or moredatabases. In embodiments, the one or more databases, as mentioned abovewith respect to step S2204, may be: internal storage 1808A, externalstorage 1808B, memory/storage 1815, system memory 1804, and/or storage1903, to name a few. The first response 2802 may be stored in accordancewith the received identification information. For example, theidentification indication may indicate an age of the users. In thisexample, the computing device 2702 may store the first response 2802 byage range (e.g. 18-25 ages grouped together, 26-40 ages groupedtogether, etc.). As another example, the first response 2802 may bestored based on whether a user is part of a specific group of users. Forexample, first responses from the first group of users associated withthe first group of user devices 2708 may be stored together.

At step S2214, the computing device (e.g. computing device 2702) maydetermine authenticity of the first response (e.g. first response 2802).In embodiments, the computing device 2702 may determine the authenticitybased on one or more of: the user information 2802A, the eventinformation 2802B, the location information 2802C, the timestamp 2802D,proximity information, previously stored authenticity ratings, and/orpreviously stored responses (which may be related or unrelated to theevent). In embodiments, the user information 2802A, the eventinformation 2802B, the location information 2802C, the timestamp 2802D,proximity information, previously stored authenticity ratings, and/orpreviously stored responses (which may be related or unrelated to theevent) may be used individually to determine the authenticity of thefirst response 2802. For example, if the user information 2802A receivedwith first response 2802 indicates that the user responding to the firstresponse 2802 is not the user associated with the user device (e.g.laptop 2708-A) the event stimulus message 2704 was transmitted to (e.g.in step S2208), the computing device 2702 may determine that the firstresponse received is unauthentic and/or has a higher probability ofbeing unauthentic. As another example, if the event information 2802Breceived with the first response 2802 contains incorrect information,the computing device 2702 may determine that the first response receivedis unauthentic and/or has a higher probability of being unauthentic(e.g. the event information 2802B may be a response to a query with ananswer the computing device 2702 already knows). As another example, ifthe location information 2802C received with the first response 2802indicates that the user was not in the area at the time of the event,the computing device 2702 may determine that the first response receivedis unauthentic and/or has a higher probability of being unauthentic. Asyet another example, if the timestamp 2802D received with the firstresponse 2802 indicates that the user waited too long before sending thefirst response 2802 (e.g. the time lag is above a predetermined time laglimit), the computing device 2702 may determine that the first responsereceived is unauthentic and/or has a higher probability of beingunauthentic. As yet another example, if proximity information receivedwith the first response 2802 indicates that the first response receivedfrom the laptop 2708-A is inconsistent with responses received from thewearable device 2708-B and the cellphone 2708-C, the computing device2702 may obtain and analyze the proximity information of the laptop2708-A as compared to the wearable device 2708-B and the cellphone2708-C. If the laptop 2708-A is outside a certain distance of thewearable device 2708-B and the cellphone 2708-C, the computing device2702 may determine that the user associated with the laptop 2708-A maynot be observing and/or experiencing the same event as one or more usersassociated with the wearable device 2708-B and the cellphone 2708-C,and, thus, the information received in the first response 2802 from thelaptop 2708-A may be unauthentic and/or have a higher probability ofbeing unauthentic. As yet another example, if the computing device 2702has a previously stored authenticity rating attached to one or moreusers, a response received by the one or more users may be determined tobe authentic or unauthentic based on the previous authenticity rating.Additionally, for example, the computing device 2702 may use previouslystored messages to determine the authenticity of the first response2802. The computing device 2702 may accomplish this by determining wordsand/or phrases typically used by the user associated with the userdevice transmitting the first response 2802. If the first response 2802does not contain words or phrases typically used and/or uses words orphrases that are not used by the user associated with the user devicetransmitting the first response 2802, the computing device 2702 maydetermine that the first response received is unauthentic and/or has ahigher probability of being unauthentic.

The above examples, if the facts are surrounding the informationreceived with the first response 2802 are the opposite (e.g. if theevent information 2802C contains correct information), the computingdevice 2702 may determine that the first response(s) received areauthentic and/or have a higher probability of being authentic.

In embodiments, one or more of the user information 2802A, the eventinformation 2802B, the location information 2802C, the timestamp 2802D,proximity information, previously stored authenticity ratings, and/orpreviously stored responses may be used in combination with one anotherto determine the authenticity of the first response 2802. For example,as mentioned above, time lag associated with the timestamp 2802D may beviewed in context with connectivity data included with the userinformation 2802A. Each piece of information received with the firstresponse 2802 may be used in combination with one another to providecontext to the information received. This context may increase theaccuracy of the authenticity rating determined by the computing device2702.

The determined authenticity of the first response 2802 may be determinedon a scale. The scale, for example, may include authenticity ratings onthe scale of: unauthentic, high probability of unauthentic, may beunauthentic or authentic, high probability of authentic, and authentic.As another example, the determined authenticity may have a numbersrating system, which may be, for example, on a 0-100 scale, where 0represents unauthentic and 100 represents authentic.

At step S2216, the computing device (e.g. computing device 2702) assignsa reliability rating to the respective user associated with the userdevice (e.g. laptop 2708-A) that transmitted the first response 2802.The reliability rating, in embodiments, may be based on one or more ofthe following: the first response 2802, any information included withinthe first response 2802 (e.g. user information 2802A, the eventinformation 2802B, the location information 2802C, the timestamp 2802D,and/or proximity information, to name a few), previously storedauthenticity ratings, and/or previously stored responses (which may berelated or unrelated to the event), and/or the authenticity determinedabove in step S2214, to name a few. For example, if the timestamp 2802Dis either below a time lag limit (e.g. indicating that the response wasautomated and not from a user) or above a time lag limit, the computingdevice 2702 may determine that the user associated with the timestamp2802D may have a reliability rating of unreliable. In embodiments, theexamples provided for authenticity in connection with the description ofS2214 may be applied similarly to reliability ratings, the descriptionsof which and examples applying herein.

In embodiments the reliability rating may be assigned, for example, byperforming one or more of the following steps: (i) assigning thereliability rating to be a reliable rating when the location information2802C is consistent with a location of the event and/or the timestamp2802D indicates acceptable delay (e.g. within a predetermined time laglimit); (ii) assigning the reliability rating to be an unreliable ratingwhen the location information 2802C is inconsistent with the location ofthe event and/or the timestamp 2802D indicates an unacceptable delay(e.g. above a predetermined time lag limit); and/or (iii) assigning thereliability rating as unreliable when event information 2802B includesincorrect information (e.g. the information is incorrect, theinformation received confirmed the occurrence of a past event elementthat was modified and/or did not take place). In embodiments, instead ofreliable and unreliable binary ratings, a graded reliability ratingscheme may be used by the computing device. The determined reliabilityrating of the user associated with the first response 2802 may bedetermined on a scale. The scale, for example, may include reliabilityratings on the scale of: unreliable, high probability of unreliable, maybe unreliable or reliable, high probability of reliable, and reliable.As another example, the determined reliability may have a numbers ratingsystem, which may be, for example, on a 0-100 scale, where 0 representsunreliable and 100 represents reliable.

At step S2218, the computing device (e.g. computing device 2702) storesin the one or more databases the reliability rating. In embodiments, theone or more databases, as mentioned above with respect to steps S2204and S2212, may be: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. The reliability rating may be stored in accordance with thereceived identification information. For example, the identificationindication may indicate an age of the users. In this example, thecomputing device 2702 may store the reliability rating by age range(e.g. 18-25 ages grouped together, 26-40 ages grouped together, etc.).As another example, the reliability rating may be stored based onwhether a user is part of a specific group of users. For example,reliability rating from the first group of users associated with thefirst group of user devices 2708 may be stored together. As yet anotherexample, the reliability rating may be stored based on the reliabilityratings. For example, reliable reliability ratings may be storedtogether while unreliable reliability ratings may be stored together.

At step S2220, the computing device (e.g. computing device 2702) mayidentify one or more reliable users based on the reliability rating. Theone or more reliable users may be one or more users associated with theone or more user devices of the first group of user devices 2708 (e.g. auser associated with the laptop 2708-A, a user associated with thewearable device 2708-B, and/or a user associated with the cellphone2708-C). In embodiments, the computing device 2702 may identify reliableusers based on whether the reliable rating meets and/or exceeds apredetermined reliability rating threshold. The reliability ratingthreshold, in embodiments, may be related to the type of reliabilityrating the computing device 2702 assigns to the one or more usersassociated with the one or more user devices of the first group of userdevices 2708. For example, if the reliability rating is a binary type ofreliability rating (e.g. two ratings, either reliable or unreliable),the predetermined reliability rating threshold may be predetermined tobe a reliable reliability rating. Thus, in this example, the computingdevice 2702 may identify every user with a reliable reliability ratingas reliable. As another example, if the reliability rating is a scaletype of reliability rating (e.g. unreliable, high probability ofunreliable, may be unreliable or reliable, high probability of reliable,and reliable), the predetermined reliability rating may be predeterminedto be high probability of reliable. Thus, in this example, the computingdevice 2702 may identify every user with either a high probability ofreliable rating or a reliable reliability rating as reliable. As yetanother example, if the reliability rating is a numbers type ofreliability rating (e.g. on a scale of 0-100 where 0 is unreliable and100 is reliable), the predetermined reliability rating may bepredetermined to be a reliability rating of 75. Thus, in this example,the computing device 2702 may identify every user with a reliabilityrating of 75 or above as reliable.

In embodiments, the computing device 2702 may identify one or morereliable users by filtering out one or more unreliable users. Similar tothe description above, in embodiments, the computing device 2702 mayidentify unreliable users based on whether the reliable rating is equalto or below a predetermined reliability rating threshold. The computingdevice 2702 may identify the unreliable users and exclude the responsesreceived by user devices associated with the unreliable users. Afterdiscounting responses from user devices associated with the unreliableusers, the computing device 2702 may analyze and/or store responses thatwere not excluded (e.g. responses from reliable users). In embodiments,the computing device 2702 may assign a reliability rating of reliable(or a rating that meets or is above the predetermined threshold) to eachuser associated with each user device that transmitted a response thatwas not excluded. The computing device 2702, in embodiments, may storethe one or more unreliable user's reliability rating (in a mannersimilar to the storage described in steps S2204, S2212, and/or S2218).The computing device 2702, in embodiments, may store the one or morereliable user's reliability rating (in a manner similar to the storagedescribed in steps S2204, S2212, and/or S2218).

In embodiments, the reliability rating of one or more users may affectthe respective weight placed on responses by the computing device 2702.In embodiments, information provided form users who have a reliablereliability rating may have more weight than information provided formusers who have an unreliable rating. In embodiments, if information froma reliable user conflicts with information from an unreliable user, thecomputing device 2702 may determine that the information from thereliable user is to be used instead of the information from theunreliable user. If, in embodiments, information from a reliable userdoes not conflict with information provided from an unreliable user, thecomputing device 2702 may determine that both pieces of information areto be selected by the computing device (e.g. in step S2222) and used inthe news reported to be generated (e.g. in step S2224). In embodiments,the computing device 2702 may flag information used from unreliablesources in any news report that is generated that contains informationfrom an unreliable user. In embodiments, the computing device 2702 maydetermine that a user has such a low reliability rating that anyinformation that provided by such a user will not be selected and/orused in a generated news report. This low rating, in embodiments, may beany reliability rating below a predetermined reliability ratingthreshold (e.g. any reliability rating below 20 (on a scale of 0-100)).The predetermined reliability rating threshold may vary based on thetype of news, type of details, and/or type of news story to begenerated. For example, if the news is related to a death of person, thereliability rating threshold may be higher. Continuing the example, thereliability rating threshold may be higher with regards to specificdetails, for example, the name of the person who died. In embodiments,the computing device 2702 may generate a news story and the news storymay be generated based on reliability rating thresholds. For example, ifa user would like as much information as possible, a user may requestthe computing device 2702 to generate a news story with a lowreliability threshold. As another example, if a user would like a newsstory with only completely reliable facts, a user may request thecomputing device 2702 to generate a news story with a high reliabilitythreshold. The weighting of information with regards reliability ratings(and the entirety of the embodiments described herein) may be applicableand thus similar to the manner in which the computing device 2702 usesand/or weights information with regards to the processes described belowin connection with FIGS. 23, 24, 25A, 25B, and 26 , the descriptionsapplying therein.

At step S2222, the computing device (e.g. computing device 2702) mayselect the responsive information associated with the one or morereliable users. In embodiments, the responsive information may refer toone or more of: user information 2802A, the event information 2802B, thelocation information 2802C, the timestamp 2802D, and/or proximityinformation, to name a few. The responsive information selected, inembodiments, may be any information contained in the first response 2802that is relevant to the event (e.g. a car accident). In embodiments, theresponsive information may be information contained in the firstresponse 2802 that is related to a specific event element (e.g. whetherfirst responders have arrived at the scene). In embodiments, theresponsive information may be extracted from the first response 2802.For example, if the first response 2802 contains responses to the eventstimulus message 2704 that provided prompts with multiple choiceresponse options, the computing device 2702 may extract the multiplechoice option associated with a user that has been assigned a highreliability rating. As another example, if the first response 2802contains responses to the event stimulus message 2704 that providedquestions where users can manually input their observations, thecomputing device 2702 may extract quotes from the first response 2802and/or analyze the text data of the first response 2802. The analysis ofthe text data of the first response, using one or more processor(s) ofthe computing device 2702, may result in the computing device 2702determining the observation within the text of the first response 2802.The determining of the text within the first response 2802 may enablethe computing device 2702 to generate a reliable observation based onthe determined observation within the text of the first response 2802.

At step S2224, the computing device (e.g. computing device 2702) maygenerate a news report (e.g. news report 3002 shown in connection withFIG. 30 ). based on the selected responsive information. Referring toFIG. 30 , the news report 3002 may include a statement “According toreliable sources, the event is occurring. Here are some reliable detailsof the event.” In embodiments, the news report 3002 may include theaforementioned reliable multiple choice option. In embodiments, the newsreport 3002 may include the aforementioned reliable quotes. Inembodiments, the news report 3002 may include the aforementionedreliable observations, which may have been generated by the computingdevice 2702 based on the reliable information extracted from thereliable first responses. Moreover, the news report 3002 may includeaudio data, image data, and/or video data that was included in the firstresponse 2802. In embodiments, the generated news report 3002 mayinclude the sources of the reliable details, which may include one ormore of above mentioned received identification information that isassociated with the reliable details. In embodiments, the generated newsreport may also provide links to websites (which provide a third partynews story of the event, which may include text, audio, and/or video)that align with the reliable details of the generated news report 3002.In those embodiments, the computing device 2702 may, using one or moreprocessor(s) and/or communications circuitry, via network 1910, maythird party analyze news reports associated with the event. The analysisof the third party news reports, may result in the computing device 2702determining the what the news reports are writing, showing, and/orstating. If the third party news report contains videos or images, theanalysis may include using digital image analysis, which may include oneor more of the following: 2D object recognition, 3D object recognition,image segmentation, motion detection (e.g. single particle tracking),video tracking, optical flow, 3D Pose Estimation, and/or automaticnumber plate recognition, to name a few. If the third party news reportcontains text, the analysis may include using text recognition analysis(e.g. using Optical Character Recognition (OCR)). The determining thewhat the third party news reports are writing, showing, and/or statingmay enable the computing device 2702 to compare the third party newsreports to the reliable details of the generated news report 3002. If athird party news report is deemed to align with the reliable details ofthe generated news report 3002, the generated news report 3002 may state“A reliable news report is located at the following website” and includea link or links to the reliable third party news reports.

In embodiments, the generated news report 3002 may include a generatedtimeline of event components within the event. For example, if there isa car accident, the reliable details of the generated news report 3002may indicate: the time of the accident, the time traffic started tobuild up, the time first responders arrived, the time first respondersleft the scene of the accident, the time the people involved in the caraccident left the scene of the accident (e.g. in an ambulance or ontheir own accord), and/or the time the car accident was cleaned up (e.g.the cars involved were towed). The timeline, and the times associatedwith the timeline may include the reliability rating of each time.Moreover, in embodiments, the timeline may include the percentage ofusers that agree on a certain time. For example, if 82% of the reliableusers stated that the car accident occurred at 6:32 PM, but 18% of thereliable users stated that the car accident occurred at 6:34 PM, thetimeline generated may include both times and note the percentages ofreliable users that agree on the time of the car accident. Inembodiments, the computing device 2702 may weigh the informationreceived by reliable users when generating the news report 3002. Forexample, if the news report 3002 includes a generated timeline, thecomputing device 2702 may include only the most reliable details, ifthere is a conflict. For example, if 82% of the reliable users statedthat the car accident occurred at 6:32 PM, but 18% of the reliable usersstated that the car accident occurred at 6:34 PM, the timeline generatedmay include a car accident time of 6:32 PM.

In embodiments, the news report 3002 may be transmitted to a pluralityof users associated with a plurality of user devices 3006. The pluralityof user devices 3006, in embodiments, may be each user device associatedwith and/or connected to the electronic computer network (e.g. network1910), which may include the first group of user devices 2708.

In embodiments, the computing device 2702, may, after a firstpredetermined period of time, generate and transmit a second stimulusmessage related to the event. The second stimulus message may betransmitted to the first group of user devices 2708. In embodiments, thesecond stimulus message may be transmitted to the plurality of userdevices 3006. In embodiments, the second stimulus message may only beprovided to the identified one or more reliable users of the first groupof users associated with the first group of user devices 2708. Inembodiments, the second stimulus message may include information relatedto the event. For example, the second stimulus message may include “Wehave received an alert that the situation at the event has changed, canyou provide further details regarding this change?” In embodiments, thesecond stimulus message may include executable machine readableinstructions to present a second message to each user that receives thestimulus message, prompting a response from each user that receives thesecond stimulus message. In embodiments, the predetermined amount oftime may be based on the type of event. In embodiments, thepredetermined amount of time may not be predetermined, and the secondmessage may be generated and transmitted as a result of a change incircumstance of the event (e.g. a new alert regarding the event).

In embodiments, after the computing device 2702 transmits the secondstimulus message, the computing device 2702 may receive a secondresponse from one or more user devices. In embodiments, the secondresponse may be received via the electronic computer network (e.g.network 1910). The second response, in embodiments, may include one ormore of the following: second response information related to the secondstimulus message, user information 2802A, event information, 2802B,location information 2802C, and/or timestamp 2802D, to name a few. Inembodiments, the second response may be similar to first response 2802described above, the description of which applying herein.

In embodiments, the computing device 2702 may store, in the one or moredatabases, the second response. In embodiments, the one or moredatabases, as mentioned above with respect to steps S2204, S2212, andS2218, may be: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. The second response rating may be stored in accordance with thereceived identification information. For example, the identificationindication may indicate an age of the users. In this example, thecomputing device 2702 may store the second response by age range (e.g.18-25 ages grouped together, 26-40 ages grouped together, etc.). Asanother example, the second response may be stored based on whether auser is part of a specific group of users. For example, second responsefrom the first group of users associated with the first group of userdevices 2708 may be stored together.

In embodiments, the computing device 2702 may determine the authenticityof the second response. In embodiments, the authenticity of the secondresponse may be determined by performing one or more of the followingsteps: (i) the computing device 2702 determining whether the secondresponse includes the second response information that corresponds toactivity in the event (e.g. using second response information related tothe second stimulus message, user information 2802A, event information,2802B, location information 2802C, and/or timestamp 2802D, to name afew); and/or (ii) the computing device 2702 determining whether thesecond response includes second response information relating to eventelements available to the respective user when the second message wasdisplayed on the user device associated with the respective user (using,second response information related to the second stimulus message, userinformation 2802A, event information, 2802B, location information 2802C,and/or timestamp 2802D, to name a few). Determining the authenticity ofthe second response may be similar to determining the authenticity ofthe first response 2802 described above, the description of whichapplying herein.

In embodiments, the computing device 2702 may update the reliabilityrating for the users that transmitted the second response. Thereliability rating may be updated by determining the reliability ratingfor each user, which may be similar to the above description ofdetermining a reliability rating, the description of which applyingherein. In embodiments, the reliability rating may be based on theauthenticity of the second response. In embodiments, a reliable ratingis assigned to a user if: the second response from the user deviceassociated with the respective user includes the second responseinformation that corresponds to the activity in the event; the secondresponse from the user device associated with the respective userincludes the second response information available to the respectiveuser of the user device associated with the respective user when thesecond message was displayed by the user device; the user received areliable rating for with regards to the first response; the userreceived a reliable rating with regards to the second response, the userreceived a reliable rating with regards to both the first and secondresponse; and/or the user received a reliable rating in past responses,to name a few. In embodiments, an unreliable rating may be assigned to auser if: the second response form the user device associated with therespective user includes the second response information that does notcorrespond to the activity in the event; if the second response from theuser device associated with the respective user includes second responseinformation that was not available to the respective user when thesecond message was displayed by the user device; the user received anunreliable rating for with regards to the first response; the userreceived an unreliable rating with regards to the second response, theuser received an unreliable rating with regards to both the first andsecond response; and/or the user received an unreliable rating in pastresponses, to name a few.

In embodiments, once the reliability rating is found for each user, thereliability rating for each user may be updated. The updated rating mayinclude only the new rating. Additionally, the reliability rating mayinclude the updated reliability rating and the first reliability rating.In embodiments, the reliability rating and the updated reliabilityrating may be transmitted to the user. The transmission of thereliability rating to the user it is associated with may include amessage stating the reliability rating(s), a history of the reliabilityrating(s), and/or whether the information was used in a generated newsreport.

In embodiments, the computing device 2702 may repeat steps S2218-S2224,based on the information received with regards to the second stimulusmessage and/or the second user response. In embodiments, the informationreceived and stored from the second user response may be used to updatethe timeline generated in connection with step S2224. The updatedtimeline may be generated in a similar manner as the original timeline,the description of which applying herein. The updated timeline mayinclude the updated information received from the second responses,which may be weighted in a similar manner as the first responses were,the description of which applying herein.

In embodiments, the process may continue and repeat steps S2206-S2224,generating more stimulus messages and receiving more responses as theevent develops and/or continues. This process may continue until theevent ends, or, in some embodiments, until a predetermined amount oftime after the event has ended.

The steps of the process described in connection with FIG. 22 , inembodiments, may be rearranged or omitted.

Example 13: Method of Determining Accuracy of a News Report

Referring now to FIG. 23 , illustrates a flow chart of acomputer-implemented process for determining accuracy of a news reportbased on information provided by one or more users of a plurality ofusers of an electronic computer network (e.g., network 1910 in FIG. 19 )in accordance with exemplary embodiments of the present invention.

The process of FIG. 23 may begin at step S2302. At step S2302, acomputing device 2702 (e.g., computer 1802 in FIG. 18 , server 1901 inFIG. 19 ) receives identification information associated with each userof the plurality of users of the electronic computer network. Theidentification information may be received by the computing device 2702in a manner similar to step S2202 described above in connection withFIGS. 22, 27, and 28 , the description of which applying herein.Moreover, the identification information of the process described inconnection with FIG. 23 may be similar to the identification informationdescribed in connection with FIGS. 22, 27, and 28 , the description ofwhich applying herein.

At step S2304, the computing device stores in one or more databases theidentification information. In embodiments, the one or more databasesmay be: internal storage 1808A, external storage 1808B, memory/storage1815, system memory 1804, and/or storage 1903, to name a few. Theidentification information may be stored in accordance with the receivedidentification information. For example, the identification indicationmay indicate an age of the users. In this example, the computing device2702 may store the identification information by age range (e.g. 18-25ages grouped together, 26-40 ages grouped together, etc.). As anotherexample, the identification information may be stored based on whether auser is part of a specific group of users. For example, identificationinformation of the first group of users associated with the first groupof user devices 2708 may be stored together. Step S2304 may be similarto step S2204 described above in connection with FIGS. 22, 27, and 28 ,the description of which applying herein.

At step S2306, a first stimulus message (e.g. news stimulus message2706) related to a news report is generated by the computing device2702. The news report, in embodiments, may be any text, audio, video, orcombination thereof, describing, showing, and/or depicting an event thathas, is, or will take place. The news report may be received by thecomputing device 2702 from a third party web site. Additionally, thenews report may be received by the computing device 2702 from one ormore users of the plurality of users 3006, where one or more usersrequests the computing device 2702 check the accuracy of a news report.Additionally, the computing device 2702 may decide which news report tocheck the accuracy of, based on one or more of the following factors:the amount of views the news report has; the amount of links the newsreport has; and/or the relative importance of the news report (e.g. ifthe news report relates to an election, or if the news report relates tovaccinations, etc., to name a few). In embodiments, the news report tobe vetted for accuracy by the computer device 2702 may be a news reportthat issued one or more days before the accuracy is to be checked and/orthe subject of the news report may be an event that occurred one or moredays prior to the computing device 2702 vetting the accuracy of the newsreport.

Referring to FIG. 27 , in embodiments, computing device 2702 maygenerate news stimulus message 2706. News stimulus message 2706, may,for example, include a message stating “A news agency is reporting anevent near your location, can you give any information regarding theevent?” In embodiments, the news stimulus message 2706 may includeexecutable machine readable instructions that allow for the computingdevice 2702 to determine when one or more user devices of the firstgroup of user devices 2708 received, opened, and/or began to respond tothe news stimulus message 2706. For example, the news stimulus message2706 may include Message Disposition Notification(s) (MDN). The newsstimulus message 2706 may be generated in response to determining that anews report of importance was just published. For example, as statedabove, if there is a news report of a fire occurring at 1234 5th Street,the computing device 2702 may generate a news stimulus message 2706 todetermine the validity of the news report of the fire occurring. Thenews stimulus message 2706 may be specific to the event happening. Forexample, news stimulus message 2706 may include text that states, “wehave received a report that a fire is happening at 1234 5th street, canyou tell us any information regarding this report?” Additionally, inembodiments, the news stimulus message 2706 may provide a link to thereport and/or an excerpt of the news report. Furthermore, inembodiments, the news stimulus message 2706 may also include userspecific information. For example, the news stimulus message 2706, maystate “Ken, we received a report that a fire is happening near you at1234 5th street, can you see it? If so, are firemen already present atthe scene?” In embodiments, news stimulus message 2706 may include aquestion regarding information that the computing device 2702 hasalready confirmed. For example, the computing device 2702 may havealready confirmed that firemen are at the scene. In this example, newsstimulus message 2706 may state “Ken, we received a report that a fireis happening near you at 1234 5th street, can you see it? If so, arefiremen already present at the scene? Is there any further details youcan tell us?” The first question, “are fireman already present at thescene” may be asked to confirm the reliability of the user (e.g. if theuser gets the answer correct, the user may be determined to be reliable,if the user gets the answer incorrect, the user may be determined to beunreliable). The second question, may be to confirm the reliability ofthe news report that is reporting of the fire. In embodiments, thecomputing device 2702 may only review answers from users that providethe correct answer to the question, “are fireman already present at thescene.”

In embodiments, the computing device 2702 may determine and/or storeevent start information. Event start information, in embodiments, mayrefer to the time at which the event (the event of the news stimulusmessage 2706) was first reported. In embodiments, event startinformation may refer to the time at which the event (the event of theevent stimulus message 2704) began. The time at which an event began mayonly be based on the reporting of the event, which may need to beauthenticated. This authentication process may be similar to the processdescribed in connection with FIGS. 22, 27, and 28 and/or the processdescribed in connection with FIG. 23 , the descriptions of whichapplying herein.

Referring back to FIG. 23 , the first stimulus message may includemachine readable instructions to present an inquiry message on the oneor more user devices of the first group of user devices. In embodiments,the inquiry message may be related to a past event element of the newsreport. For example, a past event element may be whether firemen haveresponded to a fire. The news report, in this example, would be thefire. The event element, in this example, is the presence of thefiremen. In embodiments, the inquiry message may be related to a futureevent element. For example, a future event element may be whether a fireis likely to spread to other buildings. The news report, in thisexample, would be the fire. The future event element, in this example,is whether the fire will spread. In embodiments the inquiry message maybe related to a past event element of the event that was modified or didnot take place. For example, if the computing device has determined thatinitial news report information included incorrect information regardingthe presence of firemen, the computing device 2702 may generate a firststimulus message that includes questions regarding the incorrectinformation. If the user responds correctly, then the user may bedetermined as a reliable source. If the user responds incorrectly, thenthe user may be determined as an unreliable source.

At step S2308, the computing device transmits the first stimulus messageto a first group of user devices associated with a first group of usersof the plurality of users (e.g., a device 1904, 1907 in FIG. 19 , aportable client computing device 2001 in FIGS. 20 and 21 , first groupof devices 2708 in FIG. 27 ) associated with a first group of users ofthe plurality of users. Referring back to FIG. 27 , computing device2702, may transmit the news stimulus message 2706 to the first group ofuser devices 2708. In embodiments, the news stimulus message 2706 may betransmitted over network 1910. In embodiments, the first group of userdevices 2708 may include one or more of: laptop 2708-A, wearable device2708-B, and/or cell phone 2708-C. First group of user devices 2708, inembodiments, may be associated with a first group of users of theplurality of users (e.g. plurality of users 3006 described in connectionwith FIG. 30 ). Laptop 2708-A, wearable device 2708-B, and/or cell phone2708-C may be similar to one or more of the following, the descriptionsapplying herein: device 1904, 1907 described above in connection withFIG. 19 , and/or portable client computing device 2001 described abovein connection with FIGS. 20 and 21 .

In embodiments, the first group of users may be determined usingidentification information. In embodiments, as mentioned above, theidentification may include past location information, which, asmentioned above, may include a timestamp related to the past locationinformation. The past location information and related timestamp, inembodiments, may enable the computing device 2702 to determine whereusers associated with the plurality of user devices 3006 were locatedduring a past event (e.g. an event being the subject of a news report,where the event is not currently taking place). The timestamp related tothe past location information may include a time range associated withthe location information (e.g. User 1 was at Location 2 between 3:00 PMand 4:00 PM). The past location information may, in embodiments, enablethe computing device 2702 to determine which users associated with theplurality of user devices 3006 were within a predetermined distance ofthe event which is the subject of the news report (e.g. within apredetermined radius, as described above in reference to FIGS. 22, 27,and 28 , the description of which applying herein). For example, if thenews report details event an event that occurred at a baseball stadium,the computing device 2702 may use past location information to determinewhich users were within a predetermined radius (e.g. at the baseballstadium) during the time of the event at the baseball stadium. Inembodiments, each user at the baseball stadium at the time (or timerange) of the event may be placed in a first group of users (e.g. firstgroup of user devices 2708).

In embodiments, the computing device 2702 may determine the first groupof users in a similar manner as described above in connection with FIGS.22, 27, and 28 , the description of which applying herein.

At step S2310, the computing device receives, from one or more userdevices (e.g. laptop 2708-A, wearable device 2708-B, and/or cellphone2708-C) of the first group of user devices (e.g. first group of userdevices 2708) a first response (e.g. first response 2902). Inembodiments, the first response (e.g. first response 2802) may includeone or more of the following: (i) user information specific to therespective user associated with the respective user device thattransmits the first response (e.g. user information 2902A); (ii)responsive information related to the news report (e.g. news reportinformation 2902B); (iii) location information associated with alocation of the respective user device associated with the respectiveuser (e.g. location information 2902C); (iv) a timestamp (e.g. timestamp2902D) (v) proximity information; (vi) audio data associated with theevent and/or observations of the user; (vii) image data associated withthe event and/or observations of a user; and/or (viii) video dataassociated with the event and/or observations of a user, to name a few.

User information 2902A may be similar to user information 2802Adescribed above in connection with FIGS. 22, 27, and 28 , thedescription of which applying herein. As described above, in referenceto FIGS. 22, 27, and 28 , user information 2902A may include connectioninformation. Connection information, as described above, may enable thecomputing device 2702 to accurately set an upper and/or lower time laglimit which may be used for authentication of the first response 2902and/or a determining a reliability rating of a user associated withcellphone 2708-C (e.g. the user device of the first group of userdevices 2708 which sent the first response 2902).

News report information 2902B may be information received from a userdevice (e.g. cellphone 2708-C) that is responsive to the news reportstimulus message 2706. For example, the news stimulus message 2706 maystate, as previously noted, “A news agency is reporting an event nearyour location, can you give any information regarding the event?” Thefirst response 2902 from the cellphone 2708-C, may include news reportinformation 2902B, which may responsive to the “any information”question. News report information 2902B may include information that isreceived via message (e.g. “Yes, the news report missed the fact thatthe white car caused the accident”) and/or information that is receivedvia answers to a prompt (e.g. a prompt that asks “Which car caused thecar accident”—where the prompt has a few preselected answers—“(a) thewhite car; (b) the red car; (c) I don't know; (d) both cars” and theuser selects one or more of the options presented).

Location information 2902C may be similar to location information 2802Cdescribed above in connection with FIGS. 22, 27, and 28 , thedescription of which applying herein.

Timestamp 2902D may be similar to timestamp 2802D described above inconnection with FIGS. 22, 27, and 28 , the description of which applyingherein. As described above, timestamp 2902D may enable the computingdevice 2702 to calculate a time lag associated with the first response2902. The computed time lag, in embodiments and as mentioned above, maybe compared to upper and/or lower time lag limits. The comparison of thecomputed time lag to the upper and/or lower time lag limits, asmentioned above, may be used for authentication of the first response2902 and/or a determining a reliability rating of a user associated withcellphone 2708-C (e.g. the user device of the first group of userdevices 2708 which sent the first response 2902).

Proximity information, as described herein, may be similar to theproximity information described above in connection with FIGS. 22, 27,and 28 , the description of which applying herein. As described above,proximity information may be used for determining the authenticity ofthe first response 2902 and/or determining a reliability rating of auser associated with cellphone 2708-C (e.g. the user device of the firstgroup of user devices 2708 which sent the first response 2902).

Audio data, as described herein, may be similar to the audio datadescribed above in connection with FIGS. 22, 27, and 28 , thedescription of which applying herein. As described above, audio data maybe used for determining the authenticity of the first response 2902and/or determining a reliability rating of a user associated withcellphone 2708-C (e.g. the user device of the first group of userdevices 2708 which sent the first response 2902).

Image data, as described herein, may be similar to the image datadescribed above in connection with FIGS. 22, 27, and 28 , thedescription of which applying herein. As described above, image data maybe used for determining the authenticity of the first response 2902and/or determining a reliability rating of a user associated withcellphone 2708-C (e.g. the user device of the first group of userdevices 2708 which sent the first response 2902).

Video data, as described herein, may be similar to the video datadescribed above in connection with FIGS. 22, 27, and 28 , thedescription of which applying herein. As described above, video data maybe used for determining the authenticity of the first response 2902and/or determining a reliability rating of a user associated withcellphone 2708-C (e.g. the user device of the first group of userdevices 2708 which sent the first response 2902).

At step S2312, the computing device (e.g. computing device 2702) storesthe first response (e.g. first response 2902) in the one or moredatabases. In embodiments, the one or more databases, as mentioned abovewith respect to step S2304, may be: internal storage 1808A, externalstorage 1808B, memory/storage 1815, system memory 1804, and/or storage1903, to name a few. The first response 2802 may be stored in accordancewith the received identification information. For example, theidentification indication may indicate an age of the users. In thisexample, the computing device 2702 may store the first response 2802 byage range (e.g. 18-25 ages grouped together, 26-40 ages groupedtogether, etc.). As another example, the first response 2802 may bestored based on whether a user is part of a specific group of users. Forexample, first responses from the first group of users associated withthe first group of user devices 2708 may be stored together. Step S2312may be similar to step S2212 described above in connection with FIGS.22, 27, and 28 , the description of which applying herein.

At step S2314, the computing device (e.g. computing device 2702) maydetermine authenticity of the first response (e.g. first response 2902).In embodiments, the computing device 2702 may determine the authenticitybased on one or more of: the user information 2902A, the news reportinformation 2902B, the location information 2902C, the timestamp 2902D,proximity information, previously stored authenticity ratings, and/orpreviously stored responses (which may be related or unrelated to thenews report). In embodiments, the user information 2902A, the newsreport information 2902B, the location information 2902C, the timestamp2902D, proximity information, previously stored authenticity ratings,and/or previously stored responses (which may be related or unrelated tothe news report) may be used individually to determine the authenticityof the first response 2902. For example, if the user information 2902Areceived with first response 2902 indicates that the user responding tothe first response 2902 is the user associated with the user device(e.g. laptop 2708-A) the news stimulus message 2706 was transmitted to(e.g. in step S2308), the computing device 2702 may determine that thefirst response 2902 received is authentic and/or has a higherprobability of being authentic. As another example, as mentioned above,if the news report information 2902B received with the first response2902 contains correct information with regards to a query with aconfirmed answer within the news stimulus message 2706, the computingdevice 2702 may determine that the first response received is authenticand/or has a higher probability of being authentic. As another example,if the location information 2902C received with the first response 2902indicates that the user was indeed in the area at the time of the eventthat is the subject of the news report, the computing device 2702 maydetermine that the first response 2706 received is authentic and/or hasa higher probability of being authentic. As yet another example, if thetimestamp 2902D received with the first response 2902 indicates that theuser was within the upper and/or lower limits of the time lag limit whensending the first response 2902 (e.g. the time lag is within theprescribed predetermined time lag limit(s)), the computing device 2702may determine that the first response received is authentic and/or has ahigher probability of being authentic. As yet another example, ifproximity information received with the first response 2902 indicatesthat the first response received from the cellphone 2708-C is consistentwith responses received from the wearable device 2708-B and the laptop2708-A, the computing device 2702 may obtain and analyze the proximityinformation of the cellphone 2708-C as compared to the wearable device2708-B and the laptop 2708-A. If the cellphone 2708-A is within acertain distance of the wearable device 2708-B and the laptop 2708-A,the computing device 2702 may determine that the user associated withthe cellphone 2708-C may be observing and/or experiencing the same event(which is the subject of the news report) as the one or more usersassociated with the wearable device 2708-B and the laptop 2708-A, and,thus, the information received in the first response 2902 from thecellphone 2708-C may be authentic and/or have a higher probability ofbeing authentic. As yet another example, if the computing device 2702has a previously stored authenticity rating attached to one or moreusers, a response received by the one or more users may be determined tobe authentic or unauthentic based on the previous authenticity rating.Additionally, for example, the computing device 2702 may use previouslystored messages to determine the authenticity of the first response2902. The computing device 2702 may accomplish this by determining wordsand/or phrases typically used by the user associated with the userdevice transmitting the first response 2902. If the first response 2902contains words or phrases typically used and/or does not use words orphrases that are not used by the user associated with the user devicetransmitting the first response 2902, the computing device 2702 maydetermine that the first response 2902 received is authentic and/or hasa higher probability of being authentic.

The above examples, if the facts are surrounding the informationreceived with the first response 2902 are the opposite (e.g. if the newsreport information 2902C contains incorrect information), the computingdevice 2702 may determine that the first response(s) received areunauthentic and/or have a higher probability of being unauthentic.

In embodiments, one or more of the user information 2902A, the eventinformation 2902B, the location information 2902C, the timestamp 2902D,proximity information, previously stored authenticity ratings, and/orpreviously stored responses may be used in combination with one anotherto determine the authenticity of the first response 2902. For example,as mentioned above, time lag associated with the timestamp 2902D may beviewed in context with connectivity data included with the userinformation 2902A. Each piece of information received with the firstresponse 2902 may be used in combination with one another to providecontext to the information received. This context may increase theaccuracy of the authenticity rating determined by the computing device2702.

The determined authenticity of the first response 2902 may be determinedon a scale. The scale, for example, may include authenticity ratings onthe scale of: unauthentic, high probability of unauthentic, may beunauthentic or authentic, high probability of authentic, and authentic.As another example, the determined authenticity may have a numbersrating system, which may be, for example, on a 0-100 scale, where 0represents unauthentic and 100 represents authentic.

Step S2314 may be similar to step S2214 described above in connectionwith FIGS. 22, 27, and 28 , the description of which applying herein.

At step S2316, the computing device (e.g. computing device 2702) assignsa reliability rating to the respective user associated with the userdevice (e.g. cellphone 2708-C) that transmitted the first response 2902.The reliability rating, in embodiments, may be based one or more of thefollowing: the first response 2902, any information included within thefirst response 2902 (e.g. user information 2902A, the event information2902B, the location information 2902C, the timestamp 2902D, and/orproximity information, to name a few), previously stored authenticityratings, and/or previously stored responses (which may be related orunrelated to the news report), and/or the authenticity determined abovein step S2314, to name a few. For example, if the timestamp 2902D isabove a lower time lag limit (e.g. indicating that the response wasprobably not automated) and/or below an upper time lag limit, thecomputing device 2702 may determine that the user associated with thetimestamp 2902D may have a reliability rating of reliable. Inembodiments, the examples provided for authenticity in connection withthe description of S2314 may be applied similarly to reliabilityratings, the descriptions of which and examples applying herein.

In embodiments the reliability rating may be assigned by the computingdevice 2702, for example, by performing one or more of the followingsteps: (i) assigning, by the computing device 2702, the reliabilityrating to be a reliable rating when the location information 2902C isconsistent with a location associated with the news report and thetimestamp 2902D indicates acceptable delay; (ii) assigning, by thecomputing device 2702, the reliability rating to be an unreliable ratingwhen the location information 2902C is inconsistent with the locationassociated with the news report and/or the timestamp 2902D indicates anunacceptable delay; and/or (iii) assigning the reliability rating asunreliable when the news report information 2802B includes incorrectinformation (e.g. the information is incorrect in response to a querywith a known answer, the information received confirmed the occurrenceof a past event element that was modified and/or did not take place). Inembodiments, instead of reliable and unreliable binary ratings, a gradedreliability rating scheme may be used by the computing device. Thedetermined reliability rating of the user associated with the firstresponse 2902 may be determined on a scale. The scale, for example, mayinclude reliability ratings on the scale of: unreliable, highprobability of unreliable, may be unreliable or reliable, highprobability of reliable, and reliable. As another example, thedetermined reliability may have a numbers rating system, which may be,for example, on a 0-100 scale, where 0 represents unreliable and 100represents reliable. Step S2316 may be similar to step S2216 describedabove in connection with FIGS. 22, 27, and 28 , the description of whichapplying herein.

At step S2318, the computing device (e.g. computing device 2702) storesin the one or more databases the reliability rating. In embodiments, theone or more databases, as mentioned above with respect to steps S2304and S2312, may be: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. The reliability rating may be stored in accordance with thereceived identification information. For example, the identificationindication may indicate an age of the users. In this example, thecomputing device 2702 may store the reliability rating by age range(e.g. 18-25 ages grouped together, 26-40 ages grouped together, etc.).As another example, the reliability rating may be stored based onwhether a user is part of a specific group of users. For example,reliability rating from the first group of users associated with thefirst group of user devices 2708 may be stored together. As yet anotherexample, the reliability rating may be stored based on the reliabilityratings. For example, reliable reliability ratings may be storedtogether while unreliable reliability ratings may be stored together.Step S2318 may be similar to step S2218 described above in connectionwith FIGS. 22, 27, and 28 , the description of which applying herein.

At step S2320, the computing device (e.g. computing device 2702) mayidentify one or more reliable users based on the reliability rating. Theone or more reliable users may be one or more users associated with theone or more user devices of the first group of user devices 2708 (e.g. auser associated with the laptop 2708-A, a user associated with thewearable device 2708-B, and/or a user associated with the cellphone2708-C). In embodiments, the computing device 2702 may identify reliableusers based on whether the reliable rating meets and/or exceeds apredetermined reliability rating threshold. The reliability ratingthreshold, in embodiments, may be related to the type of reliabilityrating the computing device 2702 assigns to the one or more usersassociated with the one or more user devices of the first group of userdevices 2708. For example, if the reliability rating is a binary type ofreliability rating (e.g. two ratings, reliable or unreliable), thepredetermined reliability rating threshold may be predetermined to be areliable reliability rating. Thus, in this example, the computing device2702 may identify every user with a reliable reliability rating asreliable. As another example, if the reliability rating is a scale typeof reliability rating (e.g. unreliable, high probability of unreliable,may be unreliable or reliable, high probability of reliable, andreliable), the predetermined reliability rating may be predetermined tobe high probability of reliable. Thus, in this example, the computingdevice 2702 may identify every user with either a high probability ofreliable rating or a reliable reliability rating as reliable. As yetanother example, if the reliability rating is a numbers type ofreliability rating (e.g. on a scale of 0-100 where 0 is unreliable and100 is reliable), the predetermined reliability rating may bepredetermined to be a reliability rating of 75. Thus, in this example,the computing device 2702 may identify every user with a reliabilityrating of 75 or above as reliable.

In embodiments, the computing device 2702 may identify one or morereliable users by filtering out one or more unreliable users. Similar tothe description above, in embodiments, the computing device 2702 mayidentify unreliable users based on whether the reliable rating is equalto or below a predetermined reliability rating threshold. The computingdevice 2702 may identify the unreliable users and exclude the responsesreceived by user devices associated with the unreliable users. Afterdiscounting responses from user devices associated with the unreliableusers, the computing device 2702 may analyze and/or store responses thatwere not excluded (e.g. responses from reliable users). In embodiments,the computing device 2702 may assign a reliability rating of reliable(or a rating that meets or is above the predetermined threshold) to eachuser associated with each user device that transmitted a response thatwas not excluded. The computing device 2702, in embodiments, may storethe one or more unreliable user's reliability rating (in a mannersimilar to the storage described in steps S2304, S2312, and/or S2318).The computing device 2702, in embodiments, may store the one or morereliable user's reliability rating (in a manner similar to the storagedescribed in steps S2304, S2312, and/or S2318). Step S2320 may besimilar to step S2220 described above in connection with FIGS. 22, 27,and 28 , the description of which applying herein.

At step S2322, the computing device (e.g. computing device 2702) mayselect responsive information associated with the one or more reliableusers. In embodiments, the responsive information may refer to one ormore of: user information 2902A, the news report information 2902B, thelocation information 2902C, the timestamp 2902D, and/or proximityinformation, to name a few. The responsive information selected, inembodiments, may be any information contained in the first response 2902that is relevant to the news report. In embodiments, the responsiveinformation may be information contained in the first response 2902 thatis related to a specific event element (e.g. whether first respondershave arrived at the scene). In embodiments, the responsive informationmay be extracted from the first response 2902. For example, if the firstresponse 2902 contains responses to the news report stimulus message2706 that provided prompts with multiple choice response options, thecomputing device 2702 may extract the multiple-choice option associatedwith a user that has been assigned a high reliability rating. As anotherexample, if the first response 2902 contains responses to the newsreport stimulus message 2706 that provided questions where users canmanually input their observations, the computing device 2702 may extractquotes from the first response 2902 and/or analyze the text data of thefirst response 2902. The analysis of the text data may be similar to theanalysis described above in connection with FIGS. 22, 27, and 28 , thedescription of which applying herein. Step S2322 may be similar to stepS2222 described above in connection with FIGS. 22, 27, and 28 , thedescription of which applying herein.

At step S2324, the computing device (e.g. computing device 2702)determines a news report reliability rating. In embodiments, the newsreliability rating may be determined based on at least the selectedresponsive information associated with the one or more reliable users.

In embodiments, the computing device 2702 may determine a news reportreliability rating by extracting one or more details of the news report.For exemplary purposes, the following are processes that the computingdevice 2702 may implement in order to extract details from the newsreport.

The text data of the news report may be provided to one or moreprocessor(s) (e.g. of processing unit 1803) of the computing device2702. The one or more processor(s) of the computing device 2702, inembodiments, may be capable of processing text data. In embodiments, theone or more processor(s) may include or be operatively connected to oneor more language models, which may be specific to one or more of: one ormore users associated with the plurality of user devices 3006; one ormore news reporting agencies; and/or one or more categories of newsreports (e.g. scientific news, political news, sports news, and/orbusiness news, to name a few), to name a few. The language models, inembodiments, may enable the one or more processor(s) of the computingdevice 2702 to determine one or more details of the news report. Inembodiments, the language models described herein may be stored in oneor more of the following: system memory 1804, internal storage 1808A,external storage 1808B, and/or memory/storage 1815.

For exemplary purposes, the following is a process that the computingdevice 2702 may perform for the purposes of analyzing text data. Inembodiments, the computing device 2702 may receive text data thatrepresents the text of the news report. The one or more processor(s) ofthe computing device 2702 may, in embodiments, analyze the text data.The analysis of the text data may begin with the one or moreprocessor(s) parsing out the text data into grammatical objects todetermine sentences within the text data. The grammatical objects may befurther used to parse out each sentence within the text data todetermine portions of each sentence associated with nouns, verbs,prepositions, etc. In embodiments, once each sentence is parsed out, theone or more processor(s) of the computing device 2702 may determine themeaning of each sentence. In embodiments, the one or more processor(s)of the computing device 2702 may determine that the sentence can havemore than one meaning. In those cases, the one or more processor(s) ofthe computing device 2702 may rank the more than one meanings of eachsentence, the ranking being by which meaning the one or moreprocessor(s) of the computing device 2702 has the highest likelihood ofbeing correct. In embodiments, the computing device 2702 may use one ormore language models to determine the correct meaning of the rankedmeanings.

Once the meanings of each sentence have been determined by the computingdevice 2702, the computing device may store the meanings of eachsentence. Moreover, the computing device 2702 may generate a list ofdetails associated with the news report by analyzing the meanings ofeach sentence, both individually and collectively. The list of detailsof the news report may be used to determine the news report reliabilityrating.

For exemplary purposes, the following is a process that the computingdevice 2702 may perform for the purposes of analyzing audio data. In thecase of video data, the computing device 2702 may extract the audio dataof the video data (if applicable) and perform the same process thatfollows. In embodiments, the computing device 2702 may receive audiodata that represents the audio of the news report. The one or moreprocessor(s) of the computing device 2702 may, in embodiments, analyzethe audio data. The one or more processors of the computing device 2702may be enabled to perform speech-to-text functionality on audio data. Inembodiments, the one or more processor(s) of the computing device 2702may implement any suitable computer implemented speech to text techniquemay be used to convert the received audio data into text, such asSOFTSOUND speech processing technologies available from the AutonomyCorporation, which is headquartered in Cambridge, England, UnitedKingdom.

Once the audio data has been processed by the speech-to-textfunctionality of the one or more processor(s) of the computing device2702, the resulting text data may be analyzed in a similar manner asstated above. The analysis of the text data may result with thecomputing device 2702 generating a list of details associated with thenews report. The list of details of the news report may be used todetermine the news report reliability rating.

For exemplary purposes, the following is a process that the computingdevice 2702 may perform for the purposes of analyzing image data. In thecase of video data, the computing device 2702 may extract the image dataof the video data (if applicable) and perform the same process thatfollows. In embodiments, the computing device 2702 may receive imagedata that represents an image associated with or the subject of the newsreport. The one or more processor(s) of the computing device 2702 may,in embodiments, analyze the image data. The one or more processors ofthe computing device 2702 may be enabled to perform one or more of thefollowing: 2D object recognition, 3D object recognition, imagesegmentation, motion detection (e.g. single particle tracking), videotracking, optical flow, 3D Pose Estimation, and/or automatic numberplate recognition, to name a few. The image analysis, in embodiments,may be used to determine whether an image of the news report isaccurate. In embodiments, images (and/or images within videos) may becompared to images (and/or images within videos) received in connectionwith the first response 2902.

In embodiments, the extracted details of the news report can be comparedto the responsive information received from one or more user devicesassociated with the one or more reliable users (“reliable responsiveinformation”). In embodiments, the reliable responsive information maybe analyzed by the computing device 2702 in a similar manner to theanalysis the computing device 2702 performs on the news report (e.g. forthe above text data, audio data, image data, and/or video data). Oncethe details of the news report are extracted and the reliable responsiveinformation is analyzed, the computing device 2702, in embodiments, maycompare the extracted details of the news report to the analyzedreliable responsive information.

For example, extracted details of the text data representing the text ofthe news report and/or extracted details of text data representing audio(including, if applicable, audio from video data of the news report) ofthe news report may be compared to analyzed text data representing thereliable responsive information. The text comparison (which may beperformed by the one or more processor(s) of the computing device 2702),in embodiments, may find similarities and/or differences between thenews report and the reliable responsive information. Based on the textcomparison, the computing device 2702 may determine a reliability ratingof the news report. In embodiments, the reliability rating may be basedon at least one of: the amount of similarities between the extracteddetails and the reliable responsive information; the amount ofdifferences between the extracted details and the reliable responsiveinformation; and/or the amount of conflicting information between theextracted details and the reliable responsive information, to name afew.

As another example, image data representing an image associated with orthe subject of the news report (including, if applicable, audio fromvideo data of the news report) may be compared to responsive image datarepresenting images (and images extracted from videos received byreliable users, if applicable) of the reliable responsive information.The image comparison (which may be performed by the one or moreprocessor(s) of the computing device 2702), in embodiments, may findsimilarities and/or differences between the news report and the reliableresponsive information. Based on the image comparison, the computingdevice 2702 may determine a reliability rating of the news report. Inembodiments, the reliability rating may be based on at least one of: theamount of similarities between the extracted details and the reliableresponsive information; the amount of differences between the extracteddetails and the reliable responsive information; and/or the amount ofconflicting information between the extracted details and the reliableresponsive information, to name a few.

In embodiments, the news report reliability rating may be a general newsreliability rating. A general news reliability rating may be a rating ofthe entire news report —which may state if the news report is reliable,unreliable, mostly reliable, or mostly unreliable, to name a few. Forexample, as shown in connection with FIG. 30 , the computing device 2702may generate a news reliability report 3004. In embodiments, as shown inFIG. 30 the news reliability report 3004 may state “The news reported bythe news agency has a reliability score of RELIABLE.” In embodiments,the news reliability report 3004 may include a link to the news reportgiven the reliability rating. In embodiments, the general newsreliability rating may be based on a predetermined threshold. Thepredetermined threshold may require one or more of the following for thenews report to be deemed reliable: a predetermined number ofsimilarities between the reliable responsive information and theextracted details of the news report; a predetermined percentage ofsimilarities between the reliable responsive information and theextracted details of the news report; below a certain number ofdifferences between the reliable responsive information and theextracted details of the news report; and/or below a certain number ofconflicting information between the reliable responsive information andthe extracted details of the news report, to name a few. Thepredetermined numbers and/or percentages associated with thepredetermined threshold may vary based on one or more of the following:the size of the news report, the amount of text in the news report, theamount of video in the news report, the amount of images in the newsreport, the amount of reliable responsive information, the amount ofunique reliable responsive information, the amount of images of thereliable responsive information, and/or the amount of video of thereliable responsive information, to name a few. For example, if the newsreport is large and the amount of reliable responsive information islarge, the computing device 2702 may require the analysis to turn upmore similarities and/or less conflicting information (as compared tothe following example) in order for the news report to be given areliable reliability rating. As another example if the news report issmall and/or the amount of reliable responsive information is small, thecomputing device 2702 may require the analysis to turn up lesssimilarities and/or more conflicting information (as compared to theaforementioned example) in order for the news report to be given areliable reliability rating.

In some embodiments, the general news reliability rating may be one ormore of the following: a binary reliability rating (e.g. two ratings,reliable or unreliable); a scale reliability rating; and/or a numbersreliability rating (e.g. on a scale of 0-100 where 0 is unreliable and100 is reliable), to name a few.

In embodiments, the news reliability rating may be a specific newsreliability rating. A specific news reliability rating may rate thereliability of each of the extracted details of the news report in viewof the reliable responsive information. A specific news reliabilityrating may, in embodiments, require the computing device 2702 to analyzeeach of the extracted details of the news report to the reliableresponsive information. The specific analysis, may be completed by usingthe above analysis for the general news reliability rating for eachextracted detail (e.g. a first extracted detail compared to the reliableresponsive information, a second extracted detail compared to thereliable responsive information, . . . N extracted detail compared tothe reliable responsive information). Each extracted detail (includingdetails of images, audio, and/or video), in embodiments, may receive arating similar to the general news reliability rating. In someembodiments, the specific news reliability rating may be one or more ofthe following: a binary reliability rating (e.g. two ratings, reliableor unreliable); a scale reliability rating; and/or a numbers reliabilityrating (e.g. on a scale of 0-100 where 0 is unreliable and 100 isreliable), to name a few.

In embodiments, there may be extracted details of the news report thatare not addressed by the reliable responsive information. Inembodiments, in a specific analysis, the news reliability report 3004may indicate the extracted details that were not able to be verified(e.g. there was no reliable responsive information applicable to theextracted detail). The news reliability report 3004 may indicate whichextracted details are either reliable and/or unreliable and/or indicatewhich extracted details were unable to be verified. In embodiments, thegeneral news report may consider which and how many extracted detailswere unable to be verified. For example, if too high of a percentage(and/or too many) of extracted details were unable to be verified, thegeneral news report may give the news report a reliability rating ofUNRELIABLE or UNABLE TO BE VERIFIED, for example.

In embodiments, the computing device 2702 may determine both a generalnews reliability rating and a specific news reliability rating.

In embodiments, the news reliability report 3004 may include a generatedtimeline of event components within the event which is the subject ofthe news report. For example, if there is a car accident, the reliableresponsive information and/or extracted details of the news report mayindicate: the time of the accident, the time traffic started to buildup, the time first responders arrived, the time first responders leftthe scene of the accident, the time the people involved in the caraccident left the scene of the accident (e.g. in an ambulance or ontheir own accord), and/or the time the car accident was cleaned up (e.g.the cars involved were towed). The timeline, and the times associatedwith the timeline may include the reliability rating of each time.Moreover, in embodiments, the timeline may include the percentage ofusers that agree on a certain time. For example, if 82% of the reliableusers stated that the car accident occurred at 6:32 PM, but 18% of thereliable users stated that the car accident occurred at 6:34 PM, thetimeline generated may include both times and note the percentages ofreliable users that agree on the time of the car accident. Inembodiments, the computing device 2702 may weigh the informationreceived by reliable users when generating the news reliability report3004. For example, if the news reliability report 3004 includes agenerated timeline, the computing device 2702 may include only the mostreliable details, if there is a conflict. For example, if 82% of thereliable users stated that the car accident occurred at 6:32 PM, but 18%of the reliable users stated that the car accident occurred at 6:34 PM,the timeline generated may include a car accident time of 6:32 PM. Insome embodiments, the timeline may include the reliability rating ofeach extracted detail on the timeline and/or whether the extracteddetail was verified. In embodiments, the timeline may include reliableresponsive information that was not related to any of the extracteddetails, noting that the timeline point is reliable, but not availablein the news report.

In embodiments, the news report reliability rating may be stored by thecomputing device 2702 in the one or more databases. In embodiments, theone or more databases, as mentioned above with respect to steps S2304,S2312 and S318, may be: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. The news report reliability rating may be stored in accordance withone or more of the following: stored by news agency, stored byjournalist, stored by news report reliability rating, and/or stored bynews report category, to name a few.

At step S2326, the computing device (e.g. computing device 2702) maytransmit the news report reliability rating to the plurality of usersassociated with the plurality of user devices 3006. As shown in FIG. 30, the news reliability report 3004 may generated and/or transmitted bythe computing device 2702 to the plurality of user devices 3006 whichmay be associated with the plurality of users. The plurality of userdevices 3006 may include the first group of user devices 2708.

In embodiments, the process described in connection with FIG. 23 maycontinue with the computing device 2702 transmitting a second stimulusmessage related to the news report. The second stimulus message may betransmitted after a first predetermined amount of time. The secondstimulus message, in embodiments, may be transmitted to the first groupof user devices 2708. In embodiments, the second stimulus message may betransmitted to the plurality of user devices 3006. In embodiments, thesecond stimulus message may only be provided to the identified one ormore reliable users of the first group of users associated with thefirst group of user devices 2708. In embodiments, the second stimulusmessage may be transmitted via network 1910. In embodiments, the secondstimulus message may include information related to one or more of thefollowing: extracted details that were not addressed, conflictingdetails from reliable users, an updated version of the news reportand/or an additional news report regarding the same event as theoriginal news report.

For example, the computing device 2702 may determine that too manyextracted details from the news report were either not verified or notcompletely verified. In response, the computing device 2702 may generateand transmit a second stimulus message to the one or more reliable usersof the first group of users associated with the first group of devices2708. In embodiments, the second stimulus message may include “We havemore details of the News Report we would like to verify, can you verifyif one or more of these details are accurate?”

As another example, the computing device 2702 may determine that thenews report was recently updated and published. In response, thecomputing device 2702 may generate and transmit a second stimulusmessage to the one or more reliable users of the first group of usersassociated with the first group of devices 2708. In embodiments, thesecond stimulus message may include “The News Report was recentlyupdated, can you verify if the updated details are accurate?”

As yet another example, the computing device 2702 may determine that asecond news report regarding the same event that was the subject of thenews report mentioned above in connection with steps S2302-S2326 (“firstnews report”). In embodiments, the computing device 2702 may analyzeadditional news reports to determine if a second news report regardingthe same event of the first news report was published. In embodiments,the computing device 2702 may analyze the second news report and comparethe extracted details of the second news report to the extracted detailsof the first news report to determine one or more of the following: ifthe second news report includes any new details as compared to the firstnews report; if the second news report mentions the first news report;if the second news report has details that conflict with the first newsreport; if the second news report has details that differ from the firstnews report; and/or if the second news report has the same details asthe first news report. In embodiments, if there are any new details,differing details, and/or conflicting details, the computing device 2702may determine that the second news report needs to be verified for thepurposes of updating the news report reliability rating of the firstnews report. In this example, the second stimulus message may include“We have received a second news report for the event the first newsreport covered, there are additional details within the second newsreport, can you verify if one or more of these new details areaccurate?”

In embodiments, the second stimulus message may include executablemachine-readable instructions to present a second message to each userthat receives the stimulus message, prompting a response from each userthat receives the second stimulus message. In embodiments, thepredetermined amount of time may be based on the type of news report. Inembodiments, the predetermined amount of time may not be predetermined,and the second message may be generated and transmitted as a result of achange in circumstance of the news report (e.g. a second news report ispublished, more details of the first news report need to be verified, oran update of the first news report was published, to name a few).

In embodiments, after the computing device 2702 transmits the secondstimulus message, the computing device 2702 may receive a secondresponse from one or more user devices. In embodiments, the secondresponse may be received via the electronic computer network (e.g.network 1910). The second response, in embodiments, may include one ormore of the following: second response information related to the secondstimulus message, user information 2902A, news report information,2902B, location information 2902C, and/or timestamp 2902D, to name afew. In embodiments, the second response may be similar to firstresponse 2902 described above, the description of which applying herein.

In embodiments, the computing device 2702 may store, in the one or moredatabases, the second response. In embodiments, the one or moredatabases, as mentioned above with respect to steps S3204, S2312, andS2318, may be: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. The second response rating may be stored in accordance with thereceived identification information. For example, the identificationindication may indicate an age of the users. In this example, thecomputing device 2702 may store the second response by age range (e.g.18-25 ages grouped together, 26-40 ages grouped together, etc.). Asanother example, the second response may be stored based on whether auser is part of a specific group of users. For example, second responsefrom the first group of users associated with the first group of userdevices 2708 may be stored together.

In embodiments, the computing device 2702 may determine the authenticityof the second response. In embodiments, the authenticity of the secondresponse may be determined by performing one or more of the followingsteps: (i) the computing device 2702 determining whether the secondresponse includes the second response information that corresponds to anactivity element (e.g. event element) in the news report (e.g. usingsecond response information related to the second stimulus message, userinformation 2902A, news report information, 2902B, location information2902C, and/or timestamp 2902D, to name a few); and/or (ii) the computingdevice 2702 determining whether the second response includes secondresponse information relating to information (e.g. details) relating tothe news report available to the respective user when the second messagewas displayed on the user device associated with the respective user(using, second response information related to the second stimulusmessage, user information 2902A, news report information, 2902B,location information 2902C, and/or timestamp 2902D, to name a few).Determining the authenticity of the second response may be similar todetermining the authenticity of the first response 2902 described above,the description of which applying herein.

In embodiments, the computing device 2702 may update the reliabilityrating for the users associated with user devices that transmitted thesecond response. The reliability rating may be updated by determiningthe reliability rating for each user, which may be similar to the abovedescription of determining a reliability rating, the description ofwhich applying herein. In embodiments, the reliability rating may bebased on the authenticity of the second response. In embodiments, areliable rating is assigned to a user if: the second response from theuser device associated with the respective user includes the secondresponse information that corresponds to the activity event element inthe news report; the second response from the user device associatedwith the respective user includes the second response informationavailable to the respective user of the user device associated with therespective user when the second message was displayed by the userdevice; the user received a reliable rating for with regards to thefirst response; the user received a reliable rating with regards to thesecond response; the user received a reliable rating with regards toboth the first and second response; and/or the user received a reliablerating in past responses, to name a few. In embodiments, an unreliablerating may be assigned to a user if: the second response form the userdevice associated with the respective user includes the second responseinformation that does not correspond to the activity event element inthe news report; if the second response from the user device associatedwith the respective user includes second response information that wasnot available to the respective user when the second message wasdisplayed by the user device; the user received an unreliable rating forwith regards to the first response; the user received an unreliablerating with regards to the second response; the user received anunreliable rating with regards to both the first and second response;and/or the user received an unreliable rating in past responses, to namea few.

In embodiments, once the reliability rating is found for each user, thereliability rating for each user may be updated. The updated rating mayinclude only the new rating. Additionally, the reliability rating mayinclude the updated reliability rating and the first reliability rating.In embodiments, the reliability rating and the updated reliabilityrating may be transmitted to the user. The transmission of thereliability rating to the user it is associated with may include amessage stating the reliability rating(s), a history of the reliabilityrating(s), and/or whether the information was used in a generated newsreport.

In embodiments, the computing device 2702 may repeat steps S2318-S2324,based on the information received with regards to the second stimulusmessage and/or the second user response. In embodiments, the informationreceived and stored from the second user response may be used to updatethe timeline generated in connection with step S2324. The updatedtimeline may be generated in a similar manner as the original timeline,the description of which applying herein. The updated timeline mayinclude the updated information received from the second responsesand/or the second stimulus message, which may be weighted in a similarmanner as the first responses were, the description of which applyingherein.

In embodiments, the process may continue and repeat steps S2306-S2326,generating more stimulus messages and receiving more responses as thenews report develops and/or continues. This process may continue untilthe event which is the subject of the news report ends, or, in someembodiments, until a predetermined amount of time after the event whichis the subject of the news report has ended.

The steps of the process described in connection with FIG. 23 , inembodiments, may be rearranged or omitted.

Example 14: Method of Predicting Financial Market Conditions

Referring now to FIG. 24 , an illustrative flow chart of acomputer-implemented process for predicting financial market conditionsbased on information provided by one or more users of a plurality ofusers of an electronic computer network (e.g., network 1910 in FIG. 19 )in accordance with an exemplary embodiment of the present invention isprovided.

The process of FIG. 24 may begin at step S2402. Referring to FIG. 24 ,at step S2402, a computing device (e.g., computer 1802 in FIG. 18 ,server 1901 in FIG. 19 , computing device 2702 in FIGS. 27-42 ) receivesidentification information associated with each user of a plurality ofusers of the electronic computer network. Identification information, inembodiments, may include job history data which may include userspecific current and past employment history. In embodiments,identification information may be similar to the identificationinformation described above in connection with FIGS. 22-23, and 27-30 ,the description of which applying herein. Additionally, step S2402 maybe similar to steps S2202 and S2302, described above in connection withFIG. 22 and FIG. 23 respectively, the description of which applyingherein.

The computing device 2702 may determine a first group of usersassociated with a first group of user devices (e.g. first group of userdevices 2708). For example, if the computing device 2702 is going tosend a market query regarding a specific market (e.g. bonds) once theidentification information is received, the computing device 2702 maygroup users who have a job or who have had a job that is or is relatedto the specific market. In embodiments, the computing device 2702 maydetermine the first group of users in a similar manner as describedabove in connection with FIGS. 22, 23, 27, and 28 , the description ofwhich applying herein.

A market, as used herein, may be any asset or assets which an individualand/or corporate entity can invest in. A market can include one or moreof the following: stock market (e.g. NYSE), bond market (e.g. for bonds,bills, notes, and/or certificates of deposit, to name a few), foreignexchange market (e.g. Forex), physical assets (e.g. metals, jewelry,real estate, and/or cattle, to name a few), derivatives market (e.g. foroptions, futures, and/or forwards, to name a few), annuities, and/orinvestment funds, to name a few.

At step S2404, the computing device (e.g. computing device 2702) storesin one or more databases the identification information. In embodiments,the one or more databases may be: internal storage 1808A, externalstorage 1808B, memory/storage 1815, system memory 1804, and/or storage1903, to name a few. The identification information may be stored inaccordance with the received identification information. For example,the identification indication may indicate an age of the users. In thisexample, the computing device 2702 may store the identificationinformation by age range (e.g. 18-25 ages grouped together, 26-40 agesgrouped together, etc.). As another example, the identificationinformation may be stored based on whether a user is part of a specificgroup of users. For example, identification information of the firstgroup of users associated with the first group of user devices 2708 maybe stored together. Step S2404 may be similar to step S2204 describedabove in connection with FIGS. 22, 27, and 28 , the description of whichapplying herein.

At step S2406, the computing device (e.g. computing device 2702)generates a first market query (e.g. first market query 3102). Thepurpose of the first market query 3102, in embodiments, may be todetermine the reliability of one or more users receiving the firstmarket query. In embodiments, the response to the first market query3102 may be used by the computing device 2702 to determine whether afuture market condition predicted by the user should be viewed asreliable information. Thus, in embodiments, the first market query 3102,may be related to past financial market conditions. For example,referring to FIG. 31 , the first market query 3102 may include a messagestating “Have you purchased any call options in the past quarter? If so,what call option did you purchase and how much did you spend?”

In embodiments, the past financial market conditions may include one ormore of: past price information and/or past volume information, to namea few. Past price information may refer to a price that the user haspaid for a first specific asset on a market. Past volume information mayrefer to an amount of a second specific asset that the user haspurchased. In embodiments, the first specific asset and the secondspecific asset may be the same asset.

In embodiments, the first market query 3102 may include executablemachine-readable instructions that allow for the computing device 2702to determine when one or more user devices of the first group of userdevices 2708 received, opened, and/or began to respond to the firstmarket query 3102. For example, the first market query 3102 may includeMessage Disposition Notification(s) (MDN).

In embodiments, the first market query 3102 may be generated in responseto the computing device 2702 determining that an event is occurring thatis related to a market. For example, if there is a report of a firstmarket dropping in value, the computing device 2702 may generate firstmarket query 3102 to determine which users are reliable with respect tothe first market. The first market query 3102 may be specific to theevent related to the market. For example, first market query 3102 mayinclude text that states, “we have received a report that the FirstMarket is dropping in value, do you own any assets related to the FirstMarket? If so, how much did you pay for the asset and how much of theassets do you own?” Additionally, in embodiments, the first market query3102 may provide a link to the report and/or an excerpt of the report.Furthermore, in embodiments, the first market query 3102 may alsoinclude user specific information. For example, the first market query3102, may state “Bob, we have received a report that the First Market isdropping in value, do you own any assets related to the First Market? Ifso, how much did you pay for the asset and how much of the assets do youown?” In embodiments, the first market query 3102 may include a questionregarding information that the computing device 2702 has alreadyconfirmed. For example, the computing device 2702 may have alreadyconfirmed that the First Market dropped by 3 points last quarter. Inthis example, the first market query 3102 may state “Bob, do you knowhow the First Market performed last quarter? Do you own any assetsrelated to the First Market? If so, how much did you pay for the assetand how much of the assets do you own?” The first question, “do you knowhow the First Market performed last quarter” may be asked to confirm thereliability of the user. The remaining questions, may be to confirm ifthe user is involved in the First Market, which may also be used todetermine the reliability of the user. In embodiments, the computingdevice 2702 may only review answers from users that provide the correctanswer to the question, “do you know how the First Market performed lastquarter?”

In embodiments, the computing device 2702 may determine and/or storemarket start information. Market start information, in embodiments, mayrefer to the time at which the event (the event which may be the subjectof the process described in FIG. 24 ) started occurring.

The first market query (e.g. first market query 3102) may includemachine readable instructions to present an inquiry message on the oneor more user devices of the first group of user devices 2708. Inembodiments, the inquiry message may be related to the past financialmarket condition(s) mentioned above.

In embodiments the inquiry message may be related to a past financialmarket condition(s) that were modified or did not take place. Forexample, the computing device 2702 may have already confirmed that theFirst Market dropped by 3 points last quarter. In this example, thecomputing device 2702 may generate a first market query that may state“Do you know how the First Market performed last quarter?” If the userresponds correctly, then the user may be determined as a reliablesource. If the user responds incorrectly, then the user may bedetermined as an unreliable source.

At step S2408, the computing device (e.g. computing device 2702)transmits the first market query (e.g. first market query 3102) to atleast a first group of user devices (e.g. first group of user devices2708) associated with a first group of users of the plurality of users(e.g. users associated with plurality of user devices 3006) of theelectronic computer network (e.g. network 1910). Referring to FIG. 31 ,computing device 2702, may transmit the first market query 3102 to thefirst group of user devices 2708. In embodiments, the first market query3102 may be transmitted over network 1910. In embodiments, the firstgroup of user devices 2708 may include one or more of: laptop 2708-A,wearable device 2708-B, and/or cell phone 2708-C. First group of userdevices 2708, in embodiments, may be associated with a first group ofusers of the plurality of users (e.g. plurality of users 3006 describedin connection with FIG. 30 ). The step S2408 may be similar to stepS2208 described above in connection with FIGS. 22 and 27 , thedescription of which applying herein.

At step S2410, the computing device (e.g. computing device 2702) mayreceive a first market response (e.g. First Market Response 3202). Thefirst market response, in embodiments, may be received from one or moreuser devices of the first group of user devices (e.g. first group ofuser devices 2708). Referring to FIG. 32 , in embodiments, the computingdevice 2702 may receive the first market response 3202 from a first userassociated with the laptop 2708-A and a second user associated with thecellphone 2708-C. The first market response 3202, in embodiments, mayinclude multiple responses specific to each user. For example, the firstmarket response 3202 may include a specific first user market responseand a specific second user market response. While not shown in FIG. 32 ,continuing the example, the first user market response may be receivedby the computing device 2702 at a different time than the time at whichthe computing device 2702 receives the second user market response.While it is clear in the art that messages from different electronicdevices do not need to be received at the same time, for brevity andclarity purposes, only one first market response (First Market Response3202) is shown in FIG. 32 .

In embodiments, the first market response 3202 may include one or moreof the following: (i) user information 3202A unique to the respectiveuser associated with the respective user device providing the firstmarket response; (ii) past market information 3202B related to priormarket conditions; (iii) a first timestamp 3202C; (iv) locationinformation associated with a location of the respective user deviceassociated with the respective user; (v) proximity information; (vi)audio data associated with the past financial market conditions and/orobservations of the user; (vii) image data associated with the pastfinancial market conditions and/or observations of a user; and/or (viii)video data associated with the past financial market conditions and/orobservations of a user, to name a few.

User information 3202A may be similar to user information 2802A and userinformation 2902A described above in connection with FIGS. 22, 23, 27,and 28 , the description of which applying herein. User information3202A, in embodiments, may also include one or more of the following:job history data which may include user specific current and pastemployment history (similar to the job history data described above inconnection with the identification information received in step S2402),a user account associated with a user; an e-mail address associated witha user; a name associated with a user; biometric data associated with auser; gender information of the user associated with the user device;age of the user associated with the user device; personal data of a userassociated with the user device, which is either volunteered by the useror received via access that is consented to by the user; locationinformation associated with the user device; identification informationrelated to a user device associated with a user of the plurality ofusers (e.g. metadata, device type, etc., to name a few), and/orelectronic identification (e.g. electronic identification card,electronic signature, etc., to name a few), to name a few. Furthermore,as described above in connection with FIGS. 22, 23 27, and 28, userinformation 3202A may include connection information. Connectioninformation, as described above, may enable the computing device 2702 toaccurately set an upper and/or lower time lag limit which may be usedfor authentication of the first market response 3202 and/or adetermining a reliability rating of a user associated with the laptop2708-A and/or the reliability rating of a user associated with thecellphone 2708-C (e.g. the user devices of the first group of userdevices 2708 which sent the first market response 3202).

Past market information 3202B may be information received from a userdevice (e.g. cellphone 2708-C) that is responsive to the first marketquery 3102. For example, past market information 3202B may be responsiveto a query in which the computing device 2702 may have alreadyconfirmed. As with the example stated above, the computing device 2702may have confirmed that the First Market dropped by 3 points lastquarter. In this example, the first market query 3102 may state “Bob, doyou know how the First Market performed last quarter? Do you own anyassets related to the First Market? If so, how much did you pay for theasset and how much of the assets do you own?” The first question, “doyou know how the First Market performed last quarter” may be asked toconfirm the reliability of the user. The remaining questions, may be toconfirm if the user is involved in the First Market, which may also beused to determine the reliability of the user. In embodiments,continuing the example, the past market information 3202B contained inthe first market response 3202 may be responsive to “Bob, do you knowhow the First Market performed last quarter? Do you own any assetsrelated to the First Market? If so, how much did you pay for the assetand how much of the assets do you own?” The first market response 3202from the laptop 2708-A and/or the cellphone 2708-C, may includeinformation that is received via message (e.g. “Yes, the First Marketdropped by three points last quarter. I own 100 dollars' worth of assetsin the First Market”) and/or information that is received via answers toa prompt (e.g. a prompt that asks “How did the First Market perform lastquarter”—where the prompt has a few preselected answers—“(a) Up 3Points; (b) Down 3 Points; (c) I don't know” and the user selects one ormore of the options presented).

First timestamp 3202C may be similar to timestamp 2802D and timestamp2902D described above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, firsttimestamp 3202C may enable the computing device 2702 to calculate a timelag associated with the first market response 3202. To compute the timelag, the first timestamp 3202C may include multiple time stamps (e.g. atime at which the first market query 3102 was sent, a time at which thefirst market query 3102 was opened, a time at which the first marketresponse 3202 was started, a time at which the first market response3202 was transmitted, and/or a time at which the first market response3202 was received, to name a few). The multiple times within the firsttimestamp 3202C may be used to compute the time lag. For example, thecomputing device 2702 may determine a time lag by finding the amount oftime between a first time when the first market query 3102 was opened bythe laptop 2708-A and a second time when the first market response wastransmitted by the laptop 2708-A to the computing device 2702. Asanother example, in embodiments, the computed time lag may be the timedifference between a first time which is not associated with the firsttime stamp 3202C (e.g. a time of a particular market event element ofthe past financial market conditions) and a second time which isassociated with the first time stamp 3202C (e.g. the time at which thefirst market response 3202 was received by the computing device 2702).The computed time lag, in embodiments and as mentioned above, may becompared to upper and/or lower time lag limits. The comparison of thecomputed time lag to the upper and/or lower time lag limits, asmentioned above, may be used for authentication of the first marketresponse 3202 and/or a determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which sent thefirst market response 3202). Moreover, as mentioned above, the upperand/or lower time lag limits (thresholds) may be predetermined. Thepredetermined upper and/or lower time lag limits may be determined inthe context of the previously mentioned connection information (e.g. ifthe connection is poor, the lower and/or upper limits may be increasedto account for the poor connection, if the connection is good, the lowerand/or upper limits may be decreased to account for the good connection,etc.).

Location information as described herein, may be similar to locationinformation 2802C and location information 2902C described above inconnection with FIGS. 22, 23, 27, and 28 , the description of whichapplying herein. As described above, location information may be usedfor determining the authenticity of the first market response 3202and/or determining a reliability rating of a user associated with thelaptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s) ofthe first group of user devices 2708 which transmitted the first marketresponse 3202).

Proximity information, as described herein, may be similar to theproximity information described above in connection with FIGS. 22, 23,27, and 28 , the description of which applying herein. As describedabove, proximity information may be used for determining theauthenticity of the first market response 3202 and/or determining areliability rating of a user associated with the laptop 2708-A and/orthe cellphone 2708-C (e.g. the user device(s) of the first group of userdevices 2708 which transmitted the first market response 3202).

Audio data, as described herein, may be similar to the audio datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, audio data maybe used for determining the authenticity of the first market response3202 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the firstmarket response 3202).

Image data, as described herein, may be similar to the image datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, image data maybe used for determining the authenticity of the first market response3202 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the firstmarket response 3202).

Video data, as described herein, may be similar to the video datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, video data maybe used for determining the authenticity of the first market response3202 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the firstmarket response 3202).

In embodiments, the computing device 2702 may use information withinfirst market response 3202, (including e.g. user information 3202A, pastmarket information 3202B, first timestamp 3202C, location information,proximity information, audio data, and/or image data, to name a few) todetermine reliability and/or filter out unreliable responses. Forexample, the computing device 2702 may receive the past marketinformation 3202B of first market response 3202 and the first timestamp3202C of first market response 3202. In this example, the firsttimestamp 3202C may include the began drafting first market responsetime of the first market query and the sent time of the first marketresponse. In embodiments, the computing device 2702 may compare thebegan drafting first market response time and the sent time of the firstmarket response to calculate a time lag. This time lag may be used todetermine the reliability of the user associated with a user device thattransmitted the first market response 3202. In embodiments, this timelag may be viewed in the context of the past market information 3202B.If, for example, the time lag is high, and the amount of informationcontained within the past market information 3202B is high, thecomputing device 2702 may determine that the user associated with thetransmission of the first market response 3202 is reliable and/or has ahigher probability of being reliable because the high lag time may bedue to the amount of information input by the user. If, for example, thetime lag is high and the amount of information contained within the pastmarket information 3202B is low, the computing device 2702 may determinethat the first market response 3202 is unreliable and/or has a higherprobability of being unreliable because the amount of content sent bythe user does not reflect the amount of time spent crafting the firstmarket response 3202. If, for example, the time lag is low and theamount of information contained within the past market information 3202Bis high, the computing device 2702 may determine that the first marketresponse 3202 is unreliable and/or has a higher probability of beingunreliable because a user may have not been able to send a response withthe high amount of information within the amount of time.

Referring to FIG. 24 , at step S2412, the computing device (e.g.computing device 2702) stores in the one or more databases the firstmarket response (e.g. first market response 3202) of each user device ofthe plurality of user devices of the first group of user devices (e.g.first group of user devices 2708) from which the first market response(e.g. first market response 3202) was received. In embodiments, the oneor more databases, as mentioned above with respect to step S2404, maybe: internal storage 1808A, external storage 1808B, memory/storage 1815,system memory 1804, and/or storage 1903, to name a few. Step S2412 maybe similar to step S2212 described above in connection with FIGS. 22,27, and 28 , the description of which applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first market response 3202, the computing device may determine theauthenticity of the first market response 3202. The process ofdetermining the authenticity of the first market response 3202 may besimilar to the process of authenticating the first response 2802 of FIG.28 and/or the first response 2902 of FIG. 29 , described above inconnection with FIGS. 22, 23, 27, 28, 29, and 30 , the descriptions ofwhich applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first market response 3202, the computing device may determine thereliability of the plurality of users associated with the electronicdevices (e.g. laptop 2708-A, cellphone 2708-C) that transmitted thefirst market response 3202. The process of determining the reliabilityof the plurality of users associated with the electronic devices (e.g.laptop 2708-A, cellphone 2708-C) that transmitted the first marketresponse 3202 may be similar to the process of determining thereliability of the users associated with electronic devices thattransmitted the first response 2802 of FIG. 28 and/or the first response2902 of FIG. 29 , described above in connection with FIGS. 22, 23, 27,28, 29, and 30 , the descriptions of which applying herein.

At step S2414, the computing device (e.g. computing device 2702)generates a second market query (e.g. second market query 3104). Thepurpose of the second market query, in embodiments, may be to predictfuture market conditions. In embodiments, the second market query may berelated to future market conditions. For example, referring to FIG. 31 ,the second market query 3104 may include a message stating “Do you planon purchasing a call option regarding this Asset?” In embodiments, thesecond market query 3104 may include a query relating to future marketconditions of the market that was the subject of the past marketconditions of the first market query 3102. Thus, the computing device2702 may use the first market query 3102 to determine whether a user isreliable in regard to a specific market, then use the second marketquery 3104 to ask relevant questions regarding the specific market, thequestions being directed at the reliable users in regard to the specificmarket.

In embodiments, second market query 3104 may include a query regardingfuture market conditions, which may include one or more of: future priceinformation, future volume information, future plans of purchasing,timing of the future plans of purchasing, future market predictionsand/or reasons regarding the decisions and/or predictions related to thefuture market conditions, to name a few. Future price information mayrefer to a price that the user will pay for a first specific asset on amarket. Future volume information may refer to an amount of a secondspecific asset that the user will purchase. The future plans ofpurchasing may refer to whether the user will purchase a third specificasset. Timing of the future plans of purchasing may refer to when theuser is planning on purchasing a fourth specific asset. Future marketpredictions may refer to a prediction of how a fifth specific asset willperform over a certain period of time. In embodiments, the firstspecific asset, the second specific asset, the third specific asset, thefourth specific asset and/or the fifth specific asset may be the sameasset. In embodiments, the reasons regarding the decisions and/orpredictions related to the future market conditions may refer to a querythat allows the user to give his or her reasons for their plans to ornot to purchase and/or his or her reasons for their predictions in themarket.

In embodiments, the second market query 3104 may include executablemachine-readable instructions that allow for the computing device 2702to determine when one or more user devices of the first group of userdevices 2708 received, opened, and/or began to respond to the secondmarket query 3104. For example, the second market query 3104 may includeMessage Disposition Notification(s) (MDN).

In embodiments, the second market query 3104 may be generated inresponse to the computing device 2702 determining that an event (e.g. amarket event) is occurring that is related to a market. For example, ifthere is a report of a First Market increasing in value, the computingdevice 2702 may generate the second market query 3104 to determine ifreliable users associated with the first group of user devices 2708believes the First Market will continue to increase in value. The secondmarket query 3104 may be specific to the market event related to themarket. For example, second market query 3104 may include text thatstates, “we have received a report that the First Market is increasingin value, do you believe the First Market will continue to performwell?” Additionally, in embodiments, the second market query 3104 mayprovide a link to the report and/or an excerpt of the report.Furthermore, in embodiments, the second market query 3104 may alsoinclude user specific information. For example, the second market query3104, may state “John, we have received a report that the First Marketis increasing in value, as someone who works in the First Market, do youbelieve the First Market will continue to perform well?” In embodiments,the second market query 3104 may include a question regardinginformation that the computing device 2702 has already confirmed,similar to the first market query 3102 described above, the samedescription applying herein.

In embodiments, the computing device 2702 may determine and/or storemarket start information. Market start information, in embodiments, mayrefer to the time at which the event (the event which may be the subjectof the process described in FIG. 24 ) started occurring.

The second market query (e.g. second market query 3104) may includemachine readable instructions to present an inquiry message on the oneor more user devices of the first group of user devices 2708. Inembodiments, the inquiry message may be related to the future financialmarket condition(s) mentioned above.

In embodiments the inquiry message of the second market query 3104 maybe related to financial market condition(s) that were modified or didnot take place. This may be similar to the conditions that were modifiedor did not take place that may be sent with the first market query 3102,the description of which applying herein.

At step S2416, the computing device (e.g. computing device 2702)transmits the second market query (e.g. second market query 3104) via anetwork (e.g. network 1910). In embodiments, the second market query maybe transmitted to the first group of devices (e.g. first group ofdevices 2708). In embodiments, the second market query may betransmitted to a second group of user devices. The second group of userdevices, in embodiments, may be user devices associated with one or moreof the following: authenticated users who transmitted the first marketresponses 3202 (the authentication process, in embodiments, beingperformed by the computing device 2702B), and/or users who transmittedthe first market responses 3202 where the computing device 2702 hasgiven the users a reliability rating of RELIABLE (or given a reliabilityrating which exceeds a predetermined threshold). In embodiments, thesecond market query may be transmitted to a plurality of user devices(e.g. plurality of user devices 3006) associated with the plurality ofusers of an electronic computer network (e.g. network 1910).

Referring to FIG. 31 , computing device 2702, may transmit the secondmarket query 3104 to the first group of user devices 2708. Inembodiments, the second market query 3104 may be transmitted overnetwork 1910. In embodiments, the first group of user devices 2708 mayinclude one or more of: laptop 2708-A, wearable device 2708-B, and/orcell phone 2708-C. First group of user devices 2708, in embodiments, maybe associated with a first group of users of the plurality of users(e.g. plurality of users 3006 described in connection with FIG. 30 ).The step S2416 may be similar to step S2208 described above inconnection with FIGS. 22 and 27 , the description of which applyingherein.

Referring back to FIG. 24 , At step S2418, the computing device (e.g.computing device 2702) may receive a second market response (e.g. secondmarket response 3302). The second market response, in embodiments, maybe received from one or more user devices of the first group of userdevices (e.g. first group of user devices 2708). In embodiments, thesecond market response may be received from one or more user devices ofthe aforementioned (with regards to the process of FIG. 24 ) secondgroup of user devices. Referring to FIG. 33 , the computing device 2702may receive the second market response 3302 from a first user associatedwith the laptop 2708-A and a second user associated with the cellphone2708-C. The second market response 3302, in embodiments, may includemultiple responses specific to each user. For example, the second marketresponse 3302 may include a specific first user market response and aspecific second user market response. While not shown in FIG. 33 ,continuing the example, the first user market response may be receivedby the computing device 2702 at a different time than the time at whichthe computing device 2702 receives the second user market response.While it is clear in the art that messages from different electronicdevices do not need to be received at the same time, for brevity andclarity purposes, only one second market response (second marketresponse 3302) is shown in FIG. 33 .

In embodiments, the second market response 3302 may include one or moreof the following: (i) user information 3302A unique to the respectiveuser associated with the respective user device providing the secondmarket response; (ii) future market information 3302B related to futuremarket conditions; (iii) a second timestamp 3302C; (iv) locationinformation associated with a location of the respective user deviceassociated with the respective user; (v) proximity information; (vi)audio data associated with the future financial market conditions and/orobservations of the user; (vii) image data associated with the futurefinancial market conditions and/or observations of a user; and/or (viii)video data associated with the future financial market conditions and/orobservations of a user, to name a few.

User information 3302A may be similar to user information 3202A, userinformation 2802A, and user information 2902A described above inconnection with FIGS. 22, 23, 24, 27, 28, and 32 the descriptions ofwhich applying herein.

Future market information 3302B may be information received from a userdevice (e.g. cellphone 2708-C) that is responsive to the second marketquery 3104. For example, future market information 3302B may beresponsive to a query related to a future market condition that thecomputing device 2702 is attempting to predict. For example, in responseto a market query that states “John, we have received a report that theFirst Market is increasing in value, as someone who works in the FirstMarket, do you believe the First Market will continue to perform well?”the future market information 3302B may include information that answersthe question of “do you believe the First Market will continue toperform well?” For example, the future market information 3302B mayinclude text data representing a message that states: “Yes, the FirstMarket will continue to perform well because of their recent earningsstatements.” As another example, in response to a market query thatstates “Do you plan on purchasing a call option regarding this Asset?”the future market information 3302B may include information that answersthe question “Do you plan on purchasing a call option regarding thisAsset?” For example, the future market information 3302B may includetext data representing a message that states: “Yes, I plan on purchasinga call option of the Asset this quarter.” In embodiments, informationfrom one or more users regarding whether a purchase is to be made in aspecific market may enable the computing device 2702 to make aprediction on the future performance of the specific market.Additionally, in order to gather more information relating to potentialpurchases of an asset, the query and/or future market information 3302Bmay include information regarding timing, prices, volume, and/orreasoning, to name a few.

Second timestamp 3302C may be similar to first timestamp 3202C,timestamp 2802D, and timestamp 2902D described above in connection withFIGS. 22, 23, 24, 27, 28, and 32 , the descriptions of which applyingherein.

As described above, second timestamp 3302C may enable the computingdevice 2702 to calculate a time lag associated with the second marketresponse 3302. To compute the time lag, the second timestamp 3302C mayinclude multiple time stamps (e.g. a time at which the second marketquery 3104 was sent, a time at which the second market query 3104 wasopened, a time at which the second market response 3302 was started, atime at which the second market response 3302 was transmitted, and/or atime at which the second market response 3302 was received, to name afew). The multiple times within the second timestamp 3302C may be usedto compute the time lag. For example, the computing device 2702 maydetermine a time lag by finding the amount of time between a first timewhen the second market query 3104 was opened by the laptop 2708-A and asecond time when the second market response 3302 was transmitted by thelaptop 2708-A to the computing device 2702. As another example, inembodiments, the computed time lag may be the time difference between afirst time which is not associated with the second time stamp 3302C(e.g. a time of a particular market event element of the futurefinancial market conditions) and a second time which is associated withthe second timestamp 3302C (e.g. the time at which the second marketresponse 3302 was received by the computing device 2702). The computedtime lag, in embodiments and as mentioned above, may be compared toupper and/or lower time lag limits. The comparison of the computed timelag to the upper and/or lower time lag limits, as mentioned above, maybe used for authentication of the second market response 3302 and/or adetermining a reliability rating of a user associated with the laptop2708-A and/or the cellphone 2708-C (e.g. the user device(s) of the firstgroup of user devices 2708 which sent the second market response 3302).Moreover, as mentioned above, the upper and/or lower time lag limits(thresholds) may be predetermined. The predetermined upper and/or lowertime lag limits may be determined in the context of the previouslymentioned connection information (e.g. if the connection is poor, thelower and/or upper limits may be increased to account for the poorconnection, if the connection is good, the lower and/or upper limits maybe decreased to account for the good connection, etc.).

Location information as described herein, may be similar to locationinformation 2802C and location information 2902C described above inconnection with FIGS. 22, 23, 27, and 28 , the description of whichapplying herein. As described above, location information may be usedfor determining the authenticity of the second market response 3302and/or determining a reliability rating of a user associated with thelaptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s) ofthe first group of user devices 2708 which transmitted the second marketresponse 3302).

Proximity information, as described herein, may be similar to theproximity information described above in connection with FIGS. 22, 23,27, and 28 , the description of which applying herein. As describedabove, proximity information may be used for determining theauthenticity of the second market response 3302 and/or determining areliability rating of a user associated with the laptop 2708-A and/orthe cellphone 2708-C (e.g. the user device(s) of the first group of userdevices 2708 which transmitted the second market response 3302).

Audio data, as described herein, may be similar to the audio datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, audio data maybe used for determining the authenticity of the second market response3302 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the secondmarket response 3302).

Image data, as described herein, may be similar to the image datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, image data maybe used for determining the authenticity of the second market response3302 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the secondmarket response 3302).

Video data, as described herein, may be similar to the video datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, video data maybe used for determining the authenticity of the second market response3302 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the secondmarket response 3302).

In embodiments, the computing device 2702 may use information withinsecond market response 3302, (including e.g. user information 3302A,future market information 3302B, second timestamp 3302C, locationinformation, proximity information, audio data, and/or image data, toname a few) to determine reliability of a user and/or filter outunreliable responses. The process of determining the reliability of auser and/or filtering out unreliable responses may be similar to theprocesses described above in connection with FIGS. 22, 23, 24, 27, 28,29, and 32 , the descriptions of which applying herein.

Referring to FIG. 24 , at step S2420, the computing device (e.g.computing device 2702) stores in the one or more databases the secondmarket response (e.g. second market response 3302) of each user deviceof the plurality of user devices from which the second market responsewas received (e.g. first group of user devices 2708). In embodiments,the one or more databases, as mentioned above with respect to stepS2412, may be: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. Step S2420 may be similar to step S2212 described above inconnection with FIGS. 22, 27, and 28 , the description of which applyingherein.

In embodiments, once the computing device 2702 receives and/or storesthe first market response 3202 and/or the second market response 3302,the computing device 2702 may determine the authenticity of the firstmarket response 3202 and/or the second market response 3302. The processof determining the authenticity of the first market response 3202 and/orthe second market response 3302 may be similar to the process ofauthenticating the first response 2802 of FIG. 28 and/or the firstresponse 2902 of FIG. 29 , described above in connection with FIGS. 22,23, 27, 28, 29, and 30 , the descriptions of which applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first market response 3202 and/or the second market response 3302,the computing device may determine the reliability of the plurality ofusers associated with the electronic devices (e.g. laptop 2708-A,cellphone 2708-C) that transmitted the first market response 3202 and/orthe second market response 3302. The process of determining thereliability of the plurality of users associated with the electronicdevices (e.g. laptop 2708-A, cellphone 2708-C) that transmitted thefirst market response 3202 and/or the second market response 3302 may besimilar to the process of determining the reliability of the usersassociated with electronic devices that transmitted the first response2802 of FIG. 28 and/or the first response 2902 of FIG. 29 , describedabove in connection with FIGS. 22, 23, 27, 28, 29, and 30 , thedescriptions of which applying herein.

In embodiments, if the second market response 3302 indicates that theuser associated with the user device that transmitted the second marketresponse 3302 planned on purchasing an asset in the future, thecomputing device 2702 may use that information to determine thereliability of that user for future market queries (e.g. a third marketquery, a fourth market query, . . . an N market query). For example, thecomputing device 2702 may generate an additional market query todetermine whether the user actually made the purchase that the userstated he or she was planning on making. The additional market query maybe sent after a predetermined amount of time. The predetermined amountof time, in embodiments, may be a default amount of time. Thepredetermined amount of time may be related to, or the same as, thetiming given by the user in the future market information 3302B (e.g. ifthe user stated plans to purchase Asset A within the next quarter, thepredetermined amount of time may be one day after the next quarterends). Once the additional market query is transmitted to one or moreusers that transmitted future market information 3302B, the one or moreusers may respond. If, for example, the user's response to theadditional market inquiry indicates that the user did not make thepurchase as planned, the user may be given an UNRELIABLE reliabilityrating, which may be used for future market queries. If, for example,the user's response to the additional market inquiry indicates that theuser did make the purchase as planned, the user may be given a RELIABLEreliability rating, which may be used for future market queries.

In embodiments, if the computing device 2702 has previously determinedthe reliability rating of a user, the computing device 2702 may updatethe reliability rating of the user. The updating of the reliabilityrating may be similar to the updating of the reliability ratingdescribed above in connection with the processes of FIGS. 22 and 23 ,the descriptions of which applying herein.

At step S2422, the computing device (e.g. computing device 2702)accesses at least the first market response (e.g. first market response3202) and the second market response (e.g. second market response 3302)provided by each user device of the first group of user devices (e.g.first group of user devices 2708). In embodiments, the computing device2702 may access the first market response 3202 and/or second marketresponse 3302 by receiving the stored first market response 3202 and/orthe stored second market response 3302 from one or more of: internalstorage 1808A, external storage 1808B, memory/storage 1815, systemmemory 1804, and/or storage 1903, to name a few. In embodiments, thecomputing device 2702 may access and/or receive the storedidentification information from one or more of: internal storage 1808A,external storage 1808B, memory/storage 1815, system memory 1804, and/orstorage 1903, to name a few.

At step S2424, the computing device (e.g. computing device 2702)calculates a market prediction related to the future market conditions(e.g. the subject of the second market query 3104) based on at least thefirst market response (e.g. first market response 3202) and the secondmarket response (e.g. second market response 3302). In embodiments, thecalculation may be based on the first market response (e.g. first marketresponse 3202) and the second market response (e.g. second marketresponse 3302) provided by the first group of user devices (e.g. firstgroup of user devices 2708). In embodiments, the calculation may bebased on the first market response (e.g. first market response 3202) andthe second market response (e.g. second market response 3302) providedby the second group of user devices (e.g. the aforementioned group ofuser devices associated with reliable users).

In embodiments, the calculation may use the future market information3302B that was transmitted by user devices associated with reliableusers (the reliability rating, in embodiments, being based on pastmarket information 3202B that was transmitted by user devices of thefirst group of user devices 2708). If, for example, the future marketinformation 3302B shows that users are planning on purchasing assetsrelated to the specific market (the market which is the subject of thesecond market query 3104), the computing device 2702 may determine thatthe specific market will increase in value. If, for example, the futuremarket information 3302B shows that users are planning on not purchasingand/or selling assets related to the specific market (the market whichis the subject of the second market query 3104), the computing device2702 may determine that the specific market will decrease in value. Thespecific calculations regarding future market information 3302B may bethe calculations currently used in the respective industry of thespecific market, using information received in the first market response(e.g. first market response 3202) and the second market response (e.g.second market response 3302).

In embodiments, the calculated market prediction may be based on one ormore of the following: the first market response 3202, the second marketresponse 3302, additional historical data, and/or third partyinformation (e.g. additional information from external data sources), toname a few. Additional data (e.g. additional historical data and/orthird party information) may enable the computing device 2702 to makemore accurate market predictions. The additional historical data mayinclude historical data specific to the market which was queried in thesecond market query 3104. For example, historical data may include: spotprices, futures prices, previously predicted higher prices, currentstock prices, past stock prices, and/or past performance of the market,to name a few. The third party information, in embodiments, may include:additional historical data, earnings reports, price-to-earnings ratios,products associated with the market and whether the products are comingto market soon, tariffs, taxes, legal dispute information, and/orcorporate information regarding one or more corporations within themarket, to name a few. In embodiments, the additional historical dataand/or third party information may be received by the computing device2702 via network 1910 from one or more third party vendors and/or thirdparty sources. In embodiments, the additional historical data and/orthird party information may be already stored on one or more databasesof the computing device 2702. In embodiments where the additionalinformation is already stored, the computing device 2702 may regularly(e.g. once a day, week, month, quarter, year, etc.) receive theadditional information from third party vendors and/or third partysources and store that information on one or more databases of thecomputing device 2702.

In embodiments, the computing device 2702 may implement a machinelearning technique to calculate the market prediction. In embodiments,the machine learning technique may be based on one or more of thefollowing: the first market response 3202, the second market response3302, additional historical data, and/or third party information (e.g.additional information from external data sources), to name a few. Themachine learning technique, in embodiments, may implement a machinelearning algorithm, such as supervised learning algorithms (e.g.classification supervised learning, regression supervised learning),unsupervised learning algorithms (e.g. association unsupervisedlearning, clustering unsupervised learning, dimensionality reductionunsupervised learning), reinforcement learning algorithms (e.g. throughtrial and error), semi-supervised algorithms, Naïve Bayes ClassifierAlgorithm, K Means Clustering Algorithm, Support Vector MachineAlgorithm, Apriori Algorithm, Linear Regression, Logistic Regression,Artificial Neural Networks, Random Forests, Decision Trees, and/orNearest Neighbours, to name a few. In embodiments, the machine learningtechnique may be a deep learning technique, which may be based onlearning data representations as opposed to task-specific algorithms.The deep learning technique may be supervised, semi-supervised, and/orunsupervised. In embodiments, the market prediction calculation may beperformed by using a neural network technique, which may include a deeplearning neural network technique (e.g. DNN). A DNN may be an artificialneural network with multiple layers between the input (e.g. the firstmarket response 3202 and the second market response 3302) and output(e.g. the market prediction).

In embodiments, the market prediction may be calculated using aquantitative trading strategy. The computing device 2702, inembodiments, may implement the quantitative trading strategy using thefirst market response 3202, the second market response 3302, additionalhistorical data, and/or third party information (e.g. additionalinformation from external data sources), to name a few. For example, thecomputing device 2702 may receive and/or obtain third party informationfrom more than one source, which, may include, for example, one or moreof the following: social media accounts associated with the usersassociated with the plurality of user devices 3006, social mediaaccounts associated with the users associated with the first group ofuser devices 2708, social media accounts associated with usersassociated with the aforementioned second group of user devices, and/orone or more of market research sources (e.g. Motley Fool, Investopedia,Yahoo! Finance, The Street, Wall Street Journal, MSN Money, ZacksInvestment Research, Investor Guide, Seeking Alpha, and/or OnlineBrokerages (e.g. E*TRADE, Ally Invest, and/or OptionsHouse, to name afew), to name a few).

When implementing the quantitative trading strategy, for example, thecomputing device 2702 may receive, monitor, and/or obtain market signals(e.g. market technical indicators, news, tweets, and/or other objectiveand/or subjective correlation(s), to name a few) in the context of thefirst market response 3202, and/or the second market response 3302. Inembodiments, the aforementioned market signals may be determined to berelevant and/or useful to the market prediction by one or more machinelearning algorithms being implemented by the computing device 2702 inthe context of data received in connection with the first marketresponse 3202, and/or the second market response 3302. In embodiments,as mentioned above, the machine learning algorithms may be implementedby the computing device 2702 to calculate the market prediction.

Before the current invention, machine learning algorithms (which aretypically trained by a human analyst) typically used only theaforementioned market signals to calculate market predictions. The humananalyst typically sets the relationship between the market signal and amarket trading action (e.g. quants). Unfortunately, before the currentinvention, the human trained machine learning algorithms, on their own,lead to false positives. The technical problem that continued to surfacewas that, although machine learning algorithms are typically wellequipped for interpolation (e.g. filling in missing data between twopoints or classifying if a data point is more closely associated withone market action and/or market signal or another), the machine learningalgorithms are typically bad at extrapolation based on the data. Forexample, machine learning algorithms that are implemented typically usekey words to obtain and/or receive information that may be relevant to aparticular market. Continuing the example, the machine learningalgorithms, may search for a key word associated with a particularstock. Thus, if the machine learning algorithms are searching forwhether Berkshire-Hathaway is being mentioned frequently on socialmedia, the machine learning algorithm may be receiving and/or obtaininghits on the following key words: “Berkshire-Hathaway”; “Berkshire”;“Hathaway”; “Warren Buffett”; “Warren” and “Buffett.” This type of keyword search has led to false positives, especially when other relatedkey words may be populating social media. For example, when AnneHathaway was up for an Oscar, the word “Hathaway” was all over socialmedia. The machine learning algorithms took this information with thebelief that the public was interested in Berkshire Hathaway, instead ofreading the information as irrelevant.

Exemplary embodiments may solve this technical problem with machinelearning algorithms by extrapolating data based on the incorporation ofdata received with the first market response 3202, the second marketresponse 3302, historical data, and/or additional third partyinformation. For example, the first market query 3102 may have inquiredas to the reasoning behind the uptick in the key word “Hathaway.” Aquick reliable response from one or more users would have enabled thecomputing device 2702 to have avoided the false positive associated withthe above “Hathaway” example. In embodiments, the disclosed inventionconfers an accuracy advantage because today most automated tradingand/or high frequency trading focuses on latency (e.g. the speed atwhich trading companies can send their instructions to themarket)—valuing speed over accuracy. The current invention, on the otherhand, focuses on both speed and accuracy due to the human-derivedindicators based on the data received with the first market response3202, the second market response 3302. In embodiments, this solution mayalso apply to the predictions made below in connection with Example 16,the description here applying therein.

In embodiments, once the market prediction is calculated, the computingdevice 2702 may generate a market prediction message. The marketprediction message may include one or more of the following: the marketprediction, the amount of users that sent the first market response3202, the amount of users that sent the second market response 3302, theamount of reliable users, the amount of reliable users that gaveinformation that was used in the market prediction, and/or arecommendation based on the market prediction.

In embodiments, the generated market prediction message may betransmitted, via network 1910, to one or more of: the plurality of userdevices 3006, the first group of user devices 2708, the second group ofuser devices, and/or a group of user devices associated with users whoprovided information that was used in the market prediction. Inembodiments, the market prediction message may not be sent to unreliableusers. In those embodiments, a notification message may be generated andtransmitted by the computing device 2702 to user devices associated withthe unreliable users. The notification may state why the marketprediction was not sent to the unreliable user (e.g. because yourinformation was not reliable).

The steps of the process described in connection with FIG. 24 , inembodiments, may be rearranged or omitted.

Example 15: Method of Predicting Stock Market Conditions

Referring now to FIG. 25A, an illustrative flow chart of acomputer-implemented process for predicting stock market conditionsbased on information provided by one or more users of a plurality ofusers of an electronic computer network (e.g., network 1910 in FIG. 19 )in accordance with an exemplary embodiment of the present invention.

The process of FIG. 25A may begin at step S2502. At step S2502, acomputing device (e.g., computer 1802 in FIG. 18 , server 1901 in FIG.19 , computing device 2702 in FIGS. 27-42 ) receives identificationinformation associated with each user of a plurality of users (e.g. theusers associated with the plurality of devices 3006) of the electroniccomputer network (e.g. network 1910). Identification information, inembodiments, may include job history data which may include userspecific current and past employment history. In embodiments,identification information may be similar to the identificationinformation described above in connection with FIGS. 22-23, 24 and 27-31, the description of which applying herein. Additionally, step S2502 maybe similar to steps S2202, S2302, and S2402 described above inconnection with FIGS. 22-24 respectively, the description of whichapplying herein.

The computing device 2702 may determine a first group of usersassociated with a first group of user devices (e.g. first group of userdevices 2708). For example, if the computing device 2702 is going tosend a stock market query regarding a specific stock (e.g. Apple®) oncethe identification information is received, the computing device 2702may group users who have a job or who have had a job that is or isrelated to Apple®. Additionally, for example, the computing device maygroup users who own and/or are planning to purchase Apple® stock. Inembodiments, the computing device 2702 may determine the first group ofusers in a similar manner as described above in connection with FIGS.22, 23, 24, 27, 28, and 31 the description of which applying herein.

A stock market, as used herein, may be any asset or assets which anindividual and/or corporate entity can invest in. A stock may be anystock of any business or corporation, or a set of stocks groupedtogether, an indexed metric, and/or any variation or combinationthereof.

At step S2504, the computing device (e.g. computing device 2702) storesin one or more databases the identification information. In embodiments,the one or more databases may be: internal storage 1808A, externalstorage 1808B, memory/storage 1815, system memory 1804, and/or storage1903, to name a few. The identification information may be stored inaccordance with the received identification information. For example,the identification indication may indicate an age of the users. In thisexample, the computing device 2702 may store the identificationinformation by age range (e.g. 18-25 ages grouped together, 26-40 agesgrouped together, etc.). As another example, the identificationinformation may be stored based on whether a user is part of a specificgroup of users. For example, identification information of the firstgroup of users associated with the first group of user devices 2708 maybe stored together. Step S2504 may be similar to step S2204 describedabove in connection with FIGS. 22, 27, and 28 , the description of whichapplying herein.

At step S2506, the computing device (e.g. computing device 2702)generates a first stock market query (e.g. first stock market query3402). The purpose of the first stock market query 3402, in embodiments,may be to determine the reliability of one or more users receiving thefirst stock market query. In embodiments, the response to the firststock market query 3402 may be used by the computing device 2702 todetermine whether a future stock market condition predicted by the usershould be viewed as reliable information. Thus, in embodiments, thefirst stock market query 3402, may be related to prior stock marketconditions. For example, referring to FIG. 34 , the first stock marketquery 3402 may include a message stating “Has stock from Corporation Arisen above 100 dollars a share?”

In embodiments, the prior stock market conditions may include one ormore of: past price information for stock (e.g. Stock A), volumeinformation for stock (e.g. Stock B) and/or past price/volumeinformation for a sector, to name a few. In embodiments, Stock A andStock B may be the same particular stock (e.g. Google® Stock). Pastprice information may refer to a price that the user has paid for StockA. Past price information may also refer to general past priceinformation regarding Stock A. Past volume information may refer to anamount of a Stock B that the user has purchased. Past volume informationmay also refer to general past volume information regarding Stock B.Past price/volume information for a sector may refer to past priceinformation with regards to a sector (i.e. an area of the economy inwhich businesses and/or corporations share the same or a related productor service and/or an industry or market that shares common operatingcharacteristics). Past price/volume information for a sector may referto past volume information with regards to a sector.

In embodiments, the first stock market query 3402 may include executablemachine readable instructions that allow for the computing device 2702to determine when one or more user devices of the first group of userdevices 2708 received, opened, and/or began to respond to the firststock market query 3402. For example, the first stock market query 3402may include Message Disposition Notification(s) (MDN).

In embodiments, the first stock market query 3402 may be generated inresponse to the computing device 2702 determining that an event (e.g. astock market event) is occurring that is related to a stock. Forexample, if there is a report of a Stock A dropping in value, thecomputing device 2702 may generate first stock market query 3402 todetermine which users are reliable with respect to the Stock A. Thefirst stock market query 3402 may be specific to the event. For example,first stock market query 3402 may include text that states, “we havereceived a report that Stock A is dropping in value, do you own Stock A?If so, how much did you pay for the Stock A and how much of Stock A doyou own?” Additionally, in embodiments, the first stock market query3402 may provide a link to the report and/or an excerpt of the report.Furthermore, in embodiments, the first stock market query 3402 may alsoinclude user specific information. For example, the first stock marketquery 3402, may state “Christina, we have received a report that Stock Ais dropping in value, do you own? If so, how much did you pay for StockA and how much of Stock A do you own?” In embodiments, the first stockmarket query 3402 may include a question regarding information that thecomputing device 2702 has already confirmed. For example, the computingdevice 2702 may have already confirmed that Stock A dropped by 5 dollarsa share last quarter. In this example, the first stock market query 3402may state “Christina, do you know how Stock A performed last quarter? Doyou own Stock A? If so, how much did you pay for Stock A and how much ofStock A do you own?” The first question, “do you know how Stock Aperformed last quarter?” may be asked to confirm the reliability of theuser. The remaining questions, may be to confirm if the user is involvedin the Stock A, which may also be used to determine the reliability ofthe user. For example, if the user paid too much for Stock A, then thecomputing device 2702 may determine that the user is not a reliablesource of information. As another example, if the user got a good dealfor Stock A, then the computing device 2702 may determine that the useris a reliable source of information. As yet another example, if the userowns a lot of Stock A, the computing device 2702 may determine that theuser is a reliable source of information. As yet another example, if theuser owns very little of Stock A, the computing device 2702 maydetermine that the user is not a reliable source of information. Inembodiments, the computing device 2702 may only review answers of thesecond stock market query 3404 from users that provide the correctanswer to the question, “do you know how Stock A performed lastquarter?”

In embodiments, the computing device 2702 may determine and/or storestock market start information. Stock market start information, inembodiments, may refer to the time at which the event (the event whichmay be the subject of the process described in FIG. 25A) startedoccurring.

The first stock market query (e.g. first stock market query 3402) mayinclude machine readable instructions to present an inquiry message onthe one or more user devices of the first group of user devices 2708. Inembodiments, the inquiry message may be related to the prior stockmarket condition(s) mentioned above.

In embodiments the inquiry message may be related to a past stock marketcondition(s) that were modified or did not take place. For example, thecomputing device 2702 may have already confirmed that the Stock Adropped by 5 dollars a share last quarter. In this example, thecomputing device 2702 may generate a first market query that may state“Do you know how Stock A performed last quarter?” If the user respondscorrectly, then the user may be determined as a reliable source. If theuser responds incorrectly, then the user may be determined as anunreliable source.

Step S2506 may be similar to step S2406 described above in connectionwith FIG. 24 , the description of which applying herein.

Referring back to FIG. 25A, at step S2508, the computing device (e.g.computing device 2702) transmits the first stock market query (e.g.first stock market query 3402) to at least a first group of user devices(e.g. first group of user devices 2708) associated with a first group ofusers of the plurality of users (e.g. users associated with plurality ofuser devices 3006) of the electronic computer network (e.g. network1910). Referring to FIG. 34 , computing device 2702, may transmit thefirst stock market query 3402 to the first group of user devices 2708.In embodiments, the first market query 3102 may be transmitted overnetwork 1910. In embodiments, the first group of user devices 2708 mayinclude by way of example such devices as laptop 2708-A, wearable device2708-B, and/or cell phone 2708-C. In embodiments, other devices such asdesk top computers, tablets, phablets, to name a few, may also be usedconsistent with the present invention. First group of user devices 2708,in embodiments, may be associated with a first group of users of theplurality of users (e.g. plurality of users 3006 described in connectionwith FIG. 30 ). The step S2508 may be similar to step S2208 describedabove in connection with FIGS. 22 and 27 , the description of whichapplying herein.

At step S2510, the computing device (e.g. computing device 2702) mayreceive a first stock market response (e.g. first stock market response3502). The first stock market response, in embodiments, may be receivedfrom one or more user devices of the first group of user devices (e.g.first group of user devices 2708). Referring to FIG. 35 , inembodiments, the computing device 2702 may receive the first stockmarket response 3502 from a first user associated with the laptop 2708-Aand a second user associated with the cellphone 2708-C. The first stockmarket response 3502, in embodiments, may include multiple responsesspecific to each user. For example, the first stock market response 3502may include a specific first user stock market response and a specificsecond user stock market response. While not shown in FIG. 35 ,continuing the example, the first user stock market response may bereceived by the computing device 2702 at a different time than the timeat which the computing device 2702 receives the second user stock marketresponse. While it is clear in the art that messages from differentelectronic devices do not need to be received at the same time, forbrevity and clarity purposes, only one first stock market response(first stock market response 3502) is shown in FIG. 35 .

In embodiments, the first stock market response 3502 may include one ormore of the following: (i) user information 3502A unique to therespective user associated with the respective user device providing thefirst market response; (ii) past stock market information 3502B relatedto the prior stock market conditions; (iii) a first timestamp 3502C;(iv) location information associated with a location of the respectiveuser device associated with the respective user; (v) proximityinformation; (vi) audio data associated with the prior stock marketconditions and/or observations of the user; (vii) image data associatedwith the prior stock market conditions and/or observations of a user;and/or (viii) video data associated with the prior stock marketconditions and/or observations of a user, to name a few.

User information 3502A may be similar to user information 3202A, userinformation 2802A and user information 2902A described above inconnection with FIGS. 22, 23, 24, 27, 28, 29 and 32 the description ofwhich applying herein. User information 3502A, in embodiments, may alsoinclude one or more of the following: job history data which may includeuser specific current and past employment history (similar to the jobhistory data described above in connection with the identificationinformation received in step S2502), a user account associated with auser; an e-mail address associated with a user; a name associated with auser; biometric data associated with a user; gender information of theuser associated with the user device; age of the user associated withthe user device; personal data of a user associated with the userdevice, which is either volunteered by the user or received via accessthat is consented to by the user; location information associated withthe user device; identification information related to a user deviceassociated with a user of the plurality of users (e.g. metadata, devicetype, etc., to name a few), and/or electronic identification (e.g.electronic identification card, electronic signature, etc., to name afew), to name a few. Furthermore, as described above in connection withFIGS. 22, 23, 24, 27, 28, 29, and 32 , user information 3502A mayinclude connection information. Connection information, as describedabove, may enable the computing device 2702 to accurately set an upperand/or lower time lag limit which may be used for authentication of thefirst stock market response 3502 and/or a determining a reliabilityrating of a user associated with the laptop 2708-A and/or thereliability rating of a user associated with the cellphone 2708-C (e.g.the user devices of the first group of user devices 2708 which sent thefirst stock market response 3502).

Past stock market information 3502B may be information received from auser device (e.g. cellphone 2708-C) that is responsive to the firststock market query 3402. For example, past market information 3502B maybe responsive to a query in which the computing device 2702 may havealready confirmed the answer. As with the example stated above, thecomputing device 2702 may have confirmed that Stock A dropped by 5dollars a share last quarter. In this example, the first stock marketquery 3402 may state “Bob, do you know how Stock A performed lastquarter? Do you own Stock A? If so, how much did you pay for Stock A andhow much of Stock A do you own?” The first question, “do you know howStock A performed last quarter?” may be asked to confirm the reliabilityof the user. The remaining questions, may be to confirm if the user isinvolved in Stock A and if the user is involved in Stock A, how havetheir decisions been, which may also be used to determine thereliability of the user. In embodiments, continuing the example, thepast stock market information 3502B contained in the first stock marketresponse 3502 may be responsive to “Bob, do you know how Stock Aperformed last quarter? Do you own Stock A? If so, how much did you payfor Stock A and how much of Stock A do you own?” The first stock marketresponse 3502 from the laptop 2708-A and/or the cellphone 2708-C, mayinclude information that is received via message (e.g. “Yes, the Stock Adropped by five dollars a share last quarter. I own 50 shares of StockA.”) and/or information that is received via answers to a prompt (e.g. aprompt that asks “Bob, do you know how Stock A performed lastquarter?—where the prompt has a few preselected answers—“(a) Up 3dollars a share; (b) Up 5 dollars a share; (c) I don't know” and theuser selects one or more of the options presented).

First timestamp 3502C may be similar to first timestamp 3202C, timestamp2802D, and timestamp 2902D described above in connection with FIGS. 22,23, 24, 27, 28, 29, 30, 31, and 32 , the description of which applyingherein. As described above, first timestamp 3502C may enable thecomputing device 2702 to calculate a time lag associated with the firststock market response 3502. To compute the time lag, the first timestamp3502C may include multiple time stamps (e.g. a time at which the firststock market query 3402 was sent, a time at which the first stock marketquery 3402 was opened, a time at which the first stock market response3502 was started, a time at which the first stock market response 3502was transmitted, and/or a time at which the first stock market response3502 was received, to name a few). The multiple times within the firsttimestamp 3502C may be used to compute the time lag. For example, thecomputing device 2702 may determine a time lag by finding the amount oftime between a first time when the first stock market query 3402 wasopened by the laptop 2708-A and a second time when the first stockmarket response 3502 was transmitted by the laptop 2708-A to thecomputing device 2702. As another example, in embodiments, the computedtime lag may be the time difference between a first time which is notassociated with the first time stamp 3502C (e.g. a time of a particularstock market event element of the prior stock market conditions) and asecond time which is associated with the first time stamp 3502C (e.g.the time at which the first stock market response 3502 was received bythe computing device 2702). The computed time lag, in embodiments and asmentioned above, may be compared to upper and/or lower time lag limits.The comparison of the computed time lag to the upper and/or lower timelag limits, as mentioned above, may be used for authentication of thefirst stock market response 3502 and/or a determining a reliabilityrating of a user associated with the laptop 2708-A and/or the cellphone2708-C (e.g. the user device(s) of the first group of user devices 2708which sent the first stock market response 3502). Moreover, as mentionedabove, the upper and/or lower time lag limits (thresholds) may bepredetermined. The predetermined upper and/or lower time lag limits maybe determined in the context of the previously mentioned connectioninformation (e.g. if the connection is poor, the lower and/or upperlimits may be increased to account for the poor connection, if theconnection is good, the lower and/or upper limits may be decreased toaccount for the good connection, etc.).

Location information as described herein, may be similar to locationinformation 2802C and location information 2902C described above inconnection with FIGS. 22, 23, 27, and 28 , the description of whichapplying herein. As described above, location information may be usedfor determining the authenticity of the first stock market response 3502and/or determining a reliability rating of a user associated with thelaptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s) ofthe first group of user devices 2708 which transmitted the first stockmarket response 3502).

Proximity information, as described herein, may be similar to theproximity information described above in connection with FIGS. 22, 23,27, and 28 , the description of which applying herein. As describedabove, proximity information may be used for determining theauthenticity of the first stock market response 3502 and/or determininga reliability rating of a user associated with the laptop 2708-A and/orthe cellphone 2708-C (e.g. the user device(s) of the first group of userdevices 2708 which transmitted the first stock market response 3502).

Audio data, as described herein, may be similar to the audio datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, audio data maybe used for determining the authenticity of the first stock marketresponse 3502 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe first stock market response 3502).

Image data, as described herein, may be similar to the image datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, image data maybe used for determining the authenticity of the first stock marketresponse 3502 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe first stock market response 3502).

Video data, as described herein, may be similar to the video datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, video data maybe used for determining the authenticity of the first stock marketresponse 3502 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe first stock market response 3502).

In embodiments, the computing device 2702 may use information withinfirst stock market response 3502, (including e.g. user information3502A, past stock market information 3502B, first timestamp 3502C,location information, proximity information, audio data, and/or imagedata, to name a few) to determine reliability and/or filter outunreliable responses. For example, the computing device 2702 may receivethe past stock market information 3502B of first stock market response3502 and the first timestamp 3502C of first stock market response 3502.In this example, the first timestamp 3502C may include the begandrafting first stock market response time of the first stock marketquery 3402 and the sent time of the first market response 3502. Inembodiments, the computing device 2702 may compare the began draftingfirst market response time and the sent time of the first marketresponse 3502 to calculate a time lag. This time lag may be used todetermine the reliability of the first stock market response 3502. Inembodiments, this time lag may be viewed in the context of the paststock market information 3502B. If, for example, the time lag is high,and the amount of information contained within the past stock marketinformation 3502B is high, the computing device 2702 may determine thatthe first stock market response 3502 is reliable and/or has a higherprobability of being reliable because the high lag time may be due tothe amount of information input by the user. If, for example, the timelag is high and the amount of information contained within the paststock market information 3502B is low, the computing device 2702 maydetermine that the first stock market response 3502 is unreliable and/orhas a higher probability of being unreliable because the amount ofcontent sent by the user does not reflect the amount of time spentcrafting the first stock market response 3502. If, for example, the timelag is low and the amount of information contained within the past stockmarket information 3502B is high, the computing device 2702 maydetermine that the first stock market response 3502 is unreliable and/orhas a higher probability of being unreliable because a user may have notbeen able to send a response with the high amount of information withinthe amount of time.

Referring to FIG. 25A at step S2512, the computing device (e.g.computing device 2702) stores in the one or more databases the firststock market response (e.g. first stock market response 3502) of eachuser device of the plurality of user devices of the first group of userdevices (e.g. first group of user devices 2708) from which the firststock market response (e.g. first stock market response 3502) wasreceived. In embodiments, the one or more databases, as mentioned abovewith respect to step S2504, may be: internal storage 1808A, externalstorage 1808B, memory/storage 1815, system memory 1804, and/or storage1903, to name a few. Step S2512 may be similar to step S2212 describedabove in connection with FIGS. 22, 27, and 28 , the description of whichapplying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first stock market response 3502, the computing device 2702 maydetermine the authenticity of the first stock market response 3502. Theprocess of determining the authenticity of the first stock marketresponse 3502 may be similar to the process of authenticating the firstresponse 2802 of FIG. 28 and/or the first response 2902 of FIG. 29 ,described above in connection with FIGS. 22, 23, 27, 28, 29, and 30 ,the descriptions of which applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first stock market response 3502, the computing device 2702 maydetermine the reliability of the plurality of users associated with theelectronic devices (e.g. laptop 2708-A, cellphone 2708-C) thattransmitted the first stock market response 3502. The process ofdetermining the reliability of the plurality of users associated withthe electronic devices (e.g. laptop 2708-A, cellphone 2708-C) thattransmitted the first stock market response 3502 may be similar to theprocess of determining the reliability of the users associated withelectronic devices that transmitted the first response 2802 of FIG. 28and/or the first response 2902 of FIG. 29 , described above inconnection with FIGS. 22, 23, 27, 28, 29, and 30 , the descriptions ofwhich applying herein.

At step S2514, the computing device (e.g. computing device 2702)generates a second stock market query (e.g. second stock market query3404). The purpose of the second stock market query, in embodiments, maybe to predict future stock market conditions. In embodiments, the secondstock market query may be related to future market conditions. Forexample, referring to FIG. 34 , the second stock market query 3404 mayinclude a message stating “Do you plan on purchasing stock ofCorporation A? If so, when, at what price, and how much?” Inembodiments, the second stock market query 3404 may include a queryrelating to future stock market conditions of the stock that was thesubject of the past stock market conditions of the first market query3402. Thus, the computing device 2702 may use the first market query3402 to determine whether a user is reliable in regard to a specificstock, then use the second stock market query 3404 to ask relevantquestions regarding the specific stock, the questions, in embodiments,being directed at the reliable users in regard to the specific stock.

In embodiments, second stock market query 3404 may include a queryregarding future market conditions, which may include one or more of:future stock price information, future stock volume information, futureplans of purchasing stock, timing of the future plans to purchase stock,future stock market predictions and/or reasons regarding the decisionsand/or predictions related to the future stock market conditions, toname a few. Future stock price information may refer to a price that theuser will pay for Stock A. Future volume information may refer to anamount of Stock B the user will purchase. The future plans of purchasingmay refer to whether the user will purchase Stock C and/or why (e.g.under what conditions, e.g. price/volume thresholds, news, or otherevents). Timing of the future plans of purchasing may refer to when theuser is planning on purchasing Stock D. Future market predictions mayrefer to a prediction of how a Stock E will perform over a certainperiod of time. In embodiments, Stock A, Stock B, Stock C, Stock D,and/or Stock E may be the same stock. In embodiments, the reasonsregarding the decisions and/or predictions related to the future stockmarket conditions may refer to a query that allows the user to give hisor her reasons for their plans to or not to purchase a stock and/or hisor her reasons for their predictions in the stock market.

In embodiments, the second stock market query 3404 may includeexecutable machine readable instructions that allow for the computingdevice 2702 to determine when one or more user devices of the firstgroup of user devices 2708 received, opened, and/or began to respond tothe second stock market query 3404. For example, the second stock marketquery 3404 may include Message Disposition Notification(s) (MDN).

In embodiments, the second stock market query 3404 may be generated inresponse to the computing device 2702 determining that an event (e.g. astock market event) is occurring that is related to a stock. Forexample, if there is a report of a Stock A decreasing in value, thecomputing device 2702 may generate the second stock market query 3404 todetermine if reliable users associated with the first group of userdevices 2708 believes the Stock A will continue to decrease in value.The second stock market query 3404 may be specific to the stock marketevent related to the market. For example, second stock market query 3404may include text that states, “we have received a report that Stock A isdecreasing in value, do you believe the value of Stock A will continueto fall?” Additionally, in embodiments, the second stock market query3404 may provide a link to the report and/or an excerpt of the report.Furthermore, in embodiments, the second stock market query 3404 may alsoinclude user specific information. For example, the second stock marketquery 3404, may state “Joe, we have received a report that Stock A isdecreasing in value, as someone who has purchased Stock A, do youbelieve the value of Stock A will continue to fall?” In embodiments, thesecond stock market query 3404 may include a question regardinginformation that the computing device 2702 has already confirmed,similar to the first stock market query 3402 described above, the samedescription applying herein.

In embodiments, the computing device 2702 may determine and/or storestock market start information. Stock market start information, inembodiments, may refer to the time at which the stock market event (theevent which may be the subject of the process described in FIG. 25A)started occurring.

The second stock market query (e.g. second stock market query 3404) mayinclude machine readable instructions to present an inquiry message onthe one or more user devices of the first group of user devices 2708. Inembodiments, the inquiry message may be related to the future stockmarket condition(s) mentioned above.

In embodiments the inquiry message of the second stock market query 3404may be related to stock market condition(s) that were modified or didnot take place. This may be similar to the conditions that were modifiedor did not take place that may be sent with the first market stock query3402, the description of which applying herein.

At step S2516, the computing device (e.g. computing device 2702)transmits the second stock market query (e.g. second stock market query3404) via a network (e.g. network 1910). In embodiments, the secondstock market query may be transmitted to the first group of devices(e.g. first group of devices 2708). In embodiments the second stockmarket query may be transmitted to a second group of user devices. Thesecond group of user devices, in embodiments, may be user devicesassociated with one or more of the following: authenticated users whotransmitted the first stock market responses 3502 (the authenticationprocess, in embodiments, being performed by the computing device 2702B),and/or users who transmitted the first stock market responses 3502 wherethe computing device 2702 has given the users a reliability rating ofRELIABLE (or given a reliability rating which exceeds a predeterminedthreshold). In embodiments, the second stock market query may betransmitted to a plurality of user devices (e.g. plurality of userdevices 3006) associated with the plurality of users of an electroniccomputer network (e.g. network 1910).

Referring to FIG. 34 , computing device 2702, may transmit the secondstock market query 3404 to the first group of user devices 2708. Inembodiments, the second stock market query 3404 may be transmitted overnetwork 1910. In embodiments, the first group of user devices 2708 mayinclude one or more of: laptop 2708-A, wearable device 2708-B, and/orcell phone 2708-C. First group of user devices 2708, in embodiments, maybe associated with a first group of users of the plurality of users(e.g. plurality of user devices 3006 described in connection with FIG.30 ). The step S2516 may be similar to step S2208 described above inconnection with FIGS. 22 and 27 , the description of which applyingherein.

Referring back to FIG. 25A, at step S2518, the computing device (e.g.computing device 2702) may receive a second stock market response (e.g.second stock market response 3602). The second stock market response, inembodiments, may be received from one or more user devices of the firstgroup of user devices (e.g. first group of user devices 2708). Inembodiments, the second stock market response may be received from oneor more user devices of the aforementioned (with regards to the processof FIG. 25A) second group of user devices. Referring to FIG. 36 , thecomputing device 2702 may receive the second stock market response 3602from a first user associated with the laptop 2708-A and a second userassociated with the cellphone 2708-C. The second stock market response3602, in embodiments, may include multiple responses specific to eachuser. For example, the second stock market response 3602 may include aspecific first user stock market response and a specific second userstock market response. While not shown in FIG. 36 , continuing theexample, the first user stock market response may be received by thecomputing device 2702 at a different time than the time at which thecomputing device 2702 receives the second user stock market response.While it is clear in the art that messages from different electronicdevices do not need to be received at the same time, for brevity andclarity purposes, only one second stock market response (second stockmarket response 3602) is shown in FIG. 36 .

In embodiments, the second stock market response 3602 may include one ormore of the following: (i) user information 3602A unique to therespective user associated with the respective user device providing thesecond market response; (ii) future stock market information 3602Brelated to a prediction for the future stock market conditions; (iii) asecond timestamp 3602C; (iv) location information associated with alocation of the respective user device associated with the respectiveuser; (v) proximity information; (vi) audio data associated with thefuture stock market conditions and/or observations of the user; (vii)image data associated with the future stock market conditions and/orobservations of a user; and/or (viii) video data associated with thefuture stock market conditions and/or observations of a user, to name afew.

User information 3602A may be similar to user information 3502A, userinformation 3202A, user information 2802A, and user information 2902Adescribed above in connection with FIGS. 22-35 , the descriptions ofwhich applying herein.

Future stock market information 3602B may be information received from auser device (e.g. cellphone 2708-C) that is responsive to the secondstock market query 3404. For example, future stock market information3602B may be responsive to a query related to a future stock marketcondition that the computing device 2702 is attempting to predict. Forexample, in response to a market query that states “John, we havereceived a report that Stock A is increasing in value, as someone whobuys and sells Stock A, do you believe Stock A will continue to performwell?” the future stock market information 3602B may include informationthat answers the question of “do you believe Stock A will continue toperform well?” For example, the future stock market information 3602Bmay include text data representing a message that states: “No, Stock Awill not continue to perform well because of their recent earningsstatements.” As another example, in response to a market query thatstates “Do you plan on purchasing Stock A” the future stock marketinformation 3602B may include information that answers the question “Doyou plan on purchasing Stock A?” For example, the future stock marketinformation 3602B may include text data representing a message thatstates: “No, I do not plan on purchasing Stock A.” As yet anotherexample, in response to a query that states “How high do you believe theprice of Stock A will climb, the future stock market information 3602Bmay include text data representing a message that states: “I believeStock A will climb to 101 dollars a share then fall in price.” Inembodiments, information from one or more users regarding whether apurchase is to be made of a stock may enable the computing device 2702to make a prediction on the future performance of the specific stock.Additionally, in order to gather more information relating to potentialpurchases of a stock, the query and/or future stock market information3602B may include information regarding timing, prices, volume, and/orreasoning, to name a few.

Second timestamp 3602C may be similar to first timestamp 3502C,timestamp 2802D, and timestamp 2902D described above in connection withFIGS. 22-35 , the descriptions of which applying herein.

As described above, the second timestamp 3602C may enable the computingdevice 2702 to calculate a time lag associated with the second stockmarket response 3602. To compute the time lag, the second timestamp3602C may include multiple time stamps (e.g. a time at which the secondstock market query 3404 was sent, a time at which the second stockmarket query 3404 was opened, a time at which the second stock marketresponse 3602 was started, a time at which the second stock marketresponse 3602 was transmitted, and/or a time at which the second stockmarket response 3602 was received, to name a few). The multiple timeswithin the second timestamp 3602C may be used to compute the time lag.For example, the computing device 2702 may determine a time lag byfinding the amount of time between a first time when the second stockmarket query 3404 was opened by the cellphone 2708-C and a second timewhen the second stock market response 3602 was transmitted by thecellphone 2708-C to the computing device 2702. As another example, inembodiments, the computed time lag may be the time difference between afirst time which is not associated with the second time stamp 3602C(e.g. a time of a particular stock market event element of the futurestock market conditions) and a second time which is associated with thesecond timestamp 3602C (e.g. the time at which the second stock marketresponse 3602 was received by the computing device 2702). The computedtime lag, in embodiments and as mentioned above, may be compared toupper and/or lower time lag limits. The comparison of the computed timelag to the upper and/or lower time lag limits, as mentioned above, maybe used for authentication of the second stock market response 3602and/or a determining a reliability rating of a user associated with thelaptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s) ofthe first group of user devices 2708 which sent the second stock marketresponse 3602). Moreover, as mentioned above, the upper and/or lowertime lag limits (thresholds) may be predetermined. The predeterminedupper and/or lower time lag limits may be determined in the context ofthe previously mentioned connection information (e.g. if the connectionis poor, the lower and/or upper limits may be increased to account forthe poor connection, if the connection is good, the lower and/or upperlimits may be decreased to account for the good connection, etc.).

Location information as described herein, may be similar to locationinformation 2802C and location information 2902C described above inconnection with FIGS. 22, 23, 27, and 28 , the description of whichapplying herein. As described above, location information may be usedfor determining the authenticity of the second stock market response3602 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the secondstock market response 3602).

Proximity information, as described herein, may be similar to theproximity information described above in connection with FIGS. 22, 23,27, and 28 , the description of which applying herein. As describedabove, proximity information may be used for determining theauthenticity of the second stock market response 3602 and/or determininga reliability rating of a user associated with the laptop 2708-A and/orthe cellphone 2708-C (e.g. the user device(s) of the first group of userdevices 2708 which transmitted the second stock market response 3602).

Audio data, as described herein, may be similar to the audio datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, audio data maybe used for determining the authenticity of the second stock marketresponse 3602 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe second stock market response 3602).

Image data, as described herein, may be similar to the image datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, image data maybe used for determining the authenticity of the second stock marketresponse 3602 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe second stock market response 3602).

Video data, as described herein, may be similar to the video datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, video data maybe used for determining the authenticity of the second stock marketresponse 3602 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe second stock market response 3602).

In embodiments, the computing device 2702 may use information withinsecond stock market response 3602, (including e.g. user information3602A, future stock market information 3602B, second timestamp 3602C,location information, proximity information, audio data, and/or imagedata, to name a few) to determine reliability of a user and/or filterout unreliable responses. The process of determining the reliability ofa user and/or filtering out unreliable responses may be similar to theprocesses described above in connection with FIGS. 22, 23, 24, 27, 28,29, and 32 , the descriptions of which applying herein.

Referring to FIG. 25A, at step S2520, the computing device (e.g.computing device 2702) stores in the one or more databases the secondstock market response (e.g. second stock market response 3602) of eachuser device of the plurality of user devices from which the secondmarket response was received (e.g. first group of user devices 2708). Inembodiments, the one or more databases, as mentioned above with respectto step S2512, may be: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. Step S2520 may be similar to step S2212 described above inconnection with FIGS. 22, 27, and 28 , the description of which applyingherein.

In embodiments, once the computing device 2702 receives and/or storesthe first stock market response 3502 and/or the second stock marketresponse 3602, the computing device 2702 may determine the authenticityof the first stock market response 3502 and/or the second stock marketresponse 3602. The process of determining the authenticity of the firststock market response 3502 and/or the second stock market response 3602may be similar to the process of authenticating the first response 2802of FIG. 28 and/or the first response 2902 of FIG. 29 , described abovein connection with FIGS. 22, 23, 27, 28, 29, and 30 , the descriptionsof which applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first stock market response 3502 and/or the second stock marketresponse 3602, the computing device 2702 may determine the reliabilityof the plurality of users associated with the electronic devices (e.g.laptop 2708-A, cellphone 2708-C) that transmitted the first stock marketresponse 3502 and/or the second stock market response 3602. The processof determining the reliability of the plurality of users associated withthe electronic devices (e.g. laptop 2708-A, cellphone 2708-C) thattransmitted the first stock market response 3502 and/or the second stockmarket response 3602 may be similar to the process of determining thereliability of the users associated with electronic devices thattransmitted the first response 2802 of FIG. 28 and/or the first response2902 of FIG. 29 , described above in connection with FIGS. 22, 23, 27,28, 29, and 30 , the descriptions of which applying herein.

In embodiments, if the second stock market response 3602 indicates thatthe user associated with the user device that transmitted the secondstock market response 3602 planned on purchasing a stock in the future,the computing device 2702 may use that information to determine thereliability of that user for future stock market queries (e.g. a thirdmarket query, a fourth market query, . . . , an N market query). Forexample, the computing device 2702 may generate an additional stockmarket query to determine whether the user actually made the purchasethat the user stated he or she was planning on making. The additionalmarket query may be sent after a predetermined amount of time. Thepredetermined amount of time, in embodiments, may be a default amount oftime. The predetermined amount of time may be related to, or the sameas, the timing given by the user in the future stock market information3602B (e.g. if the user stated plans to purchase Stock A within the nextquarter, the predetermined amount of time may be one day after the nextquarter ends). Once the additional market query is transmitted to one ormore users that transmitted future stock market information 3602B, theone or more users may respond via one or more electronic devicesassociated with the one or more users. If, for example, the user'sresponse to the additional market inquiry indicates that the user didnot make the purchase as planned, the user may be given an UNRELIABLEreliability rating, which may be used for future stock market queries.If, for example, the user's response to the additional market inquiryindicates that the user did make the purchase as planned, the user maybe given a RELIABLE reliability rating, which may be used for futurestock market queries.

In embodiments, if the computing device 2702 has previously determinedthe reliability rating of a user, the computing device 2702 may updatethe reliability rating of the user based on the reliability ratingassociated with the first stock market response 3502, the second stockmarket response 3602, and/or any additional responses to additionalstock market queries. The updating of the reliability rating may besimilar to the updating of the reliability rating described above inconnection with the processes of FIGS. 22 and 23 , the descriptions ofwhich applying herein.

At step S2522, the computing device accesses at least the first stockmarket response and the second stock market response provided by eachuser device of the first group of user devices, the computing device(e.g. computing device 2702) accesses at least the first stock marketresponse (e.g. first stock market response 3502) and the second stockmarket response (e.g. second stock market response 3602) provided byeach user device of the first group of user devices (e.g. first group ofuser devices 2708). In embodiments, the computing device 2702 may accessthe first stock market response 3502 and/or the second stock marketresponse 3602 by receiving the stored first stock market response 3502and/or the stored second stock market response 3602 from one or more of:internal storage 1808A, external storage 1808B, memory/storage 1815,system memory 1804, and/or storage 1903, to name a few. In embodiments,the computing device 2702 may access and/or receive the storedidentification information from one or more of: internal storage 1808A,external storage 1808B, memory/storage 1815, system memory 1804, and/orstorage 1903, to name a few.

At step S2524, the computing device (e.g. computing device 2702)calculates a stock market prediction related to the future stock marketconditions based on at least the first stock market response and secondstock market response provided by the first group of user devices. Inembodiments, the stock market prediction may be based on one or more ofthe following: the first stock market response (e.g. first stock marketresponse 3502) the second stock market response (e.g. second stockmarket response 3602), additional historical data, and/or third partyinformation (e.g. additional information from external data sources), toname a few. In embodiments, the calculation may be based on the firststock market response (e.g. first stock market response 3502) and thesecond stock market response (e.g. second stock market response 3602)provided by the first group of user devices (e.g. first group of userdevices 2708). In embodiments, the calculation may be based on the firststock market response (e.g. first stock market response 3502) and thesecond stock market response (e.g. second stock market response 3602)provided by the second group of user devices (e.g. the aforementionedgroup of user devices associated with reliable users).

In embodiments, the calculation may use the future stock marketinformation 3602B that was transmitted by user devices associated withreliable users (the reliability rating, in embodiments, being based onpast stock market information 3502B that was transmitted by user devicesof the first group of user devices 2708). If, for example, the futurestock market information 3602B shows that users are planning onpurchasing Stock A (the stock which is the subject of the second stockmarket query 3404), the computing device 2702 may determine that Stock Awill increase in value. If, for example, the future stock marketinformation 3602B shows that users are planning on not purchasing and/orselling Stock A, the computing device 2702 may determine that Stock Awill decrease in value. The specific calculations regarding future stockmarket information 3602B may be the calculations currently used in therespective industry of the stock associated with the stock marketprediction, using information received in the first stock marketresponse (e.g. first stock market response 3502) and the second stockmarket response (e.g. second stock market response 3602).

In embodiments, as mentioned above, the calculated stock marketprediction may be based on one or more of the following: the first stockmarket response 3502, the second stock market response 3602, additionalhistorical data, and/or third party information (e.g. additionalinformation from external data sources), to name a few. Additional data(e.g. additional historical data and/or third party information) mayenable the computing device 2702 to make more accurate stock marketpredictions. The additional historical data may include historical dataspecific to the industry of the stock and/or specific to the stock whichwas queried in the second stock market query 3404. For example,historical data may include: previous open stock prices, previous highstock prices, previous low stock prices, spot prices, futures prices,previously predicted higher prices, current stock prices, past stockprices, past volumes and/or past performance of the market related tothe stock, to name a few. The third party information, in embodiments,may include: additional historical data, earnings reports,price-to-earnings ratios, products associated with the market andwhether the products are coming to market soon, tariffs, taxes, legaldispute information, and/or corporate information regarding one or morecorporations associated with the stock and/or the industry related tothe stock, to name a few. In embodiments, the additional historical dataand/or third party information may be received by the computing device2702 via network 1910 from one or more third party vendors and/or thirdparty sources. In embodiments, the additional historical data and/orthird party information may be already stored on one or more databasesof the computing device 2702. In embodiments where the additionalinformation is already stored, the computing device 2702 may regularly(e.g. once a day, week, month, quarter, year, etc.) receive theadditional information from third party vendors and/or third partysources and store that information on one or more databases of thecomputing device 2702.

In embodiments, the computing device 2702 may implement a machinelearning technique to calculate the stock market prediction. Inembodiments, the machine learning technique may be based on one or moreof the following: the first stock market response 3502, the second stockmarket response 3602, additional historical data, and/or third partyinformation (e.g. additional information from external data sources), toname a few. The machine learning technique, in embodiments, mayimplement a machine learning algorithm, such as supervised learningalgorithms (e.g. classification supervised learning, regressionsupervised learning), unsupervised learning algorithms (e.g. associationunsupervised learning, clustering unsupervised learning, dimensionalityreduction unsupervised learning), reinforcement learning algorithms(e.g. through trial and error), semi-supervised algorithms, Naïve BayesClassifier Algorithm, K Means Clustering Algorithm, Support VectorMachine Algorithm, Apriori Algorithm, Linear Regression, LogisticRegression, Artificial Neural Networks, Random Forests, Decision Trees,and/or Nearest Neighbors, to name a few. In embodiments, the machinelearning technique may be a deep learning technique, which may be basedon learning data representations as opposed to task-specific algorithms.The deep learning technique may be supervised, semi-supervised, and/orunsupervised. In embodiments, the stock market prediction calculationmay be performed by using a neural network technique, which may includea deep learning neural network technique (e.g. DNN). A DNN may be anartificial neural network with multiple layers between the input (e.g.the first stock market response 3402 and the second stock marketresponse 3602) and output (e.g. the stock market prediction).

In embodiments, once the stock market prediction is calculated, thecomputing device 2702 may generate a stock market prediction message.The stock market prediction message may include one or more of thefollowing: the stock market prediction, the amount of users that sentthe first stock market response 3502, the amount of users that sent thesecond stock market response 3602, the amount of reliable users, theamount of reliable users that gave information that was used in thestock market prediction, and/or a recommendation based on the stockmarket prediction.

In embodiments, the generated stock market prediction message may betransmitted, via network 1910, to one or more of: the plurality of userdevices 3006, the first group of user devices 2708, the second group ofuser devices, and/or a group of user devices associated with users whoprovided information that was used in the stock market prediction. Inembodiments, the stock market prediction message may not be sent tounreliable users. In those embodiments, a notification message may begenerated and transmitted by the computing device 2702 to user devicesassociated with the unreliable users. The notification may state why thestock market prediction was not sent to the unreliable user (e.g.because your information was not reliable).

The steps of the process described in connection with FIG. 25A, inembodiments, may be rearranged or omitted.

In embodiments, the process of predicting stock market conditions maycontinue with the computing device (e.g. computing device 2702) furtherinclude the step of detecting, by the computing device, a tradingpattern. The trading pattern, in embodiments, may be detected inaccordance with the process illustrated by the flow chart of FIG. 25B,which may be performed with the following steps in accordance with anexemplary embodiment of the present invention.

As shown in the process illustrated by FIG. 25B, the process describedin connection with FIG. 25A may continue at a step S2500. At step S2500,a computing device (e.g. computing device 2702) may detect a tradingpattern. A trading pattern may refer to a price fluctuation of one ormore of: a stock (or commodity), market, and/or economy, to name a few.For example, a trading pattern may indicate that Stock A is trending upin value. A trade pattern, in embodiments as used herein, may includeinformation regarding one or more of: price, volume, a time associatedwith the price, a time associated with the volume, and/or a timeassociated with both the price and volume, to name a few. For example, atrading pattern may show that Stock A was valued at 300 dollars in Juneof 2017. As another example, a trading pattern may show that 5,000shares of Stock A were owned in June of 2017.

Detecting a trading pattern may, in embodiments, begin at step S2552.Referring now to FIG. 25B, at step S2552, the computing device (e.g.computing device 2702) generates a third stock market query related topast transactions. The purpose of the third stock market query 3702, inembodiments, may be to determine and/or detect a trading pattern of aspecific stock (e.g. Google® stock) and/or industry. An industry, asused in this application, may refer to any industry, including one ormore of the following: aerospace, agriculture, arms, fishing, chemical,computer, construction, defense, education, electrical power, energy,entertainment, financial services, food, fruit production, health care,hospitality, information, insurance, internet, manufacturing (e.g.automotive, electronics, pulp and paper, steel, ship building, etc.),mass media (e.g. film, music, news, publishing, world wide web, etc.),mining, petroleum, pharmaceutical, service, software,telecommunications, and/or service, to name a few. In embodiments, theresponse(s) to the second stock market query 3502 may be used by thecomputing device 2702 to determine whether a trading pattern may existwith regards to a specific stock.

Thus, in embodiments, the third stock market query 3702, may be relatedto past transactions. In embodiments, the past transactions may be stockand/or industry specific. For example, referring to FIG. 37 , thirdstock market query 3702 may include text data representing a messagethat states, “Please provide the following information regarding StockA: (A) Stock Price Information; (B) Did you Buy or Sell; (C) QuantityInformation; (D) Timing of Transaction.” In embodiments, as describedabove, stimulus questions and/or queries can be guide users to providingspecific data by limiting the options of the response to the stimulusquestion and/or query (e.g. by using multiple choice options). Inembodiments, as shown in FIG. 37 , the stimulus questions and/or queriesmay be presented in a manner where the user must fill out a blank form.In embodiments, the stimulus questions and/or queries may includeexecutable instructions that do not allow a user device to transmit aresponse to the stimulus questions and/or queries unless all of theparts of the form are filled (this also may be true with multiple choiceselections or any response to a stimulus question and/or query). Inembodiments, the stimulus questions and/or queries may includeexecutable instructions that allow a user device to transmit a responseto the stimulus questions and/or queries even if all of the parts of theform are not filled out (this also may be true with multiple choiceselections or any response to a stimulus question and/or query). Inembodiments, if all of the form (or all of the questions containedwithin a stimulus question and/or query) is not filled out, thecomputing device 2702 may determine that the user does not know theanswer to the specific question that was not filled out. In embodiments,not answer a question may affect the authenticity and/or reliabilityrating of the user associated with the device that is transmitting theresponse. For example, if a response to the third stock market query3702 may provide information that states the user bought Stock A at 500dollars a share in June of 2015. Continuing the example, if thecomputing device 2702 has already confirmed that the price of Stock Afluctuated between 200 dollars and 300 dollars during June of 2015, thecomputing device 2702 may determine that the user is unreliable. Theunreliable reliability rating may, for example, be assigned to the userbecause either the user paid too much for Stock A or it was not possibleto buy Stock A at that price during that time. The information regardingthe Stock A price and time period, in this example, may be received fromthird party sources and/or third party vendors. Additionally, theinformation may have already been previously stored (similar tohistorical data and additional information described above).

In embodiments, the past transactions may include one or more of: pastpurchases of stocks, past purchases of commodities, past purchases ofassets, past selling of stocks, past selling of commodities, and/or pastselling of assets, to name a few.

In embodiments, the third stock market query 3702 may include executablemachine readable instructions that allow for the computing device 2702to determine when one or more user devices of the first group of userdevices 2708 received, opened, and/or began to respond to the thirdstock market query 3702. For example, the third stock market query 3702may include Message Disposition Notification(s) (MDN).

In embodiments, the third stock market query 3702 may be generated inresponse to the computing device 2702 receiving a plurality of secondstock market responses from a plurality of users associated with thefirst group of user devices 2708. For example, if the computing device2702 receives more than a predetermined number of second stock marketresponses. The predetermined number, in embodiments, may be based on oneor more of the following: a number of users who responded to the secondstock market query 3404, a number of users who received a reliablereliability rating that responded to the second stock market query 3404,a number of users who responded to the second stock market query 3404 ascompared to the number of stock holders of the stock (the stock beingthe subject of the second stock market query), and/or a number ofconsistent second market responses, to name a few.

In embodiments, the third stock market query 3702 may be generated inresponse to the computing device 2702 receiving a plurality ofconsistent second stock market responses from a plurality of usersassociated with the first group of user devices 2708. For example, if amajority of users are planning on buying Stock A within the nextquarter, the computing device 2708 may detect that a trading pattern isforming, and thus initiate the process described in connection with FIG.25A.

The third stock market query (e.g. third stock market query 3702) mayinclude machine readable instructions to present an inquiry message onthe one or more user devices of the first group of user devices 2708. Inembodiments, the inquiry message may be related to the priortransactions mentioned above.

In embodiments the inquiry message may be related to a prior transactionthat was modified or did not take place. For example, the computingdevice 2702 may have already confirmed that the Stock A was valued at400 dollars a share last quarter. In this example, the computing device2702 may generate a third market query that may state “How much did youpay for Stock A last quarter?” If the user responds correctly (e.g. theprice the user alleged to pay is possible), then the user may bedetermined as a reliable source. If the user responds incorrectly (e.g.the price the user alleged to pay is not possible), then the user may bedetermined as an unreliable source.

Step S2552 may be similar to step S2506 described above in connectionwith FIG. 25A, the description of which applying herein. The third stockmarket query 3702, in embodiments, may be similar to the first stockmarket query 3402 described above in connection with FIGS. 25A and 34-36, the description of which applying herein.

Referring back to FIG. 25B, at step S2554, the computing devicetransmits the third stock market query (e.g. third stock market query3702) to the first group of user devices (e.g. first group of userdevices 2708) associated with a first group of users of the plurality ofusers (e.g. users associated with plurality of user devices 3006) of theelectronic computer network (e.g. network 1910). Referring to FIG. 37 ,the computing device 2702, may transmit the third stock market query3702 to the first group of user devices 2708. In embodiments, the thirdmarket query 3702 may be transmitted over network 1910. In embodiments,the first group of user devices 2708 may include one or more of: laptop2708-A, wearable device 2708-B, and/or cell phone 2708-C. First group ofuser devices 2708, in embodiments, may be associated with a first groupof users of the plurality of users (e.g. plurality of users 3006described in connection with FIG. 30 ). The step S2554 may be similar tostep S2208 described above in connection with FIGS. 22 and 27 , thedescription of which applying herein.

Referring back to FIG. 25B, at step S2556, the computing device (e.g.computing device 2702) may receive a third stock market response (e.g.third stock market response 3802). The second market response, inembodiments, may be received from one or more user devices of the firstgroup of user devices (e.g. first group of user devices 2708). Inembodiments, the second market response may be received from one or moreuser devices of the aforementioned (with regards to the process of FIGS.24 and 25A) second group of user devices. Referring to FIG. 38 , thecomputing device 2702 may receive the third market response 3802 from afirst user associated with the laptop 2708-A and a second userassociated with the cellphone 2708-C. The third market response 3802, inembodiments, may include multiple responses specific to each user. Forexample, the third stock market response 3802 may include a specificfirst user stock market response and a specific second user marketresponse. While not shown in FIG. 38 , continuing the example, the firstuser stock market response may be received by the computing device 2702at a different time than the time at which the computing device 2702receives the second user stock market response. While it is clear in theart that messages from different electronic devices do not need to bereceived at the same time, for brevity and clarity purposes, only onethird stock market response (third stock market response 3802) is shownin FIG. 38 .

The computing device 2702 may receive the third stock market response3802 from at least a plurality of user devices of the first group ofuser devices 2708. In embodiments, the third stock market response 3802includes: (i) user information 3802A unique to the respective userassociated with the respective user device providing the third stockmarket response; (ii) stock ID information 3802B for a particular stock;(iii) stock price information 3802C for the particular stock; (iv)buy/sell data information 3802D for the particular stock; and (v)quantity information 3802E for the particular stock; (vi) timinginformation 3802F with respect to the buy/sell data information 3802D;(vii) timestamp information; (viii) location information associated witha location of the respective user device associated with the respectiveuser; (ix) proximity information; (x) audio data associated with theparticular stock and/or observations related to the particular stock;(xi) image data associated with the particular stock and/or observationsrelated to the particular stock; and/or (xii) video data associated withthe particular stock and/or observations related to the particularstock, to name a few.

User information 3802A may be similar to user information 3602A, userinformation 3502A, user information 3302A, user information 3202A, userinformation 2802A, and user information 2902A described above inconnection with FIGS. 22-25A and 26-36 , the descriptions of whichapplying herein.

Stock ID information 3802B may be information that relates to theidentification of one or more of: a particular stock, stocks, particularcommodity, commodities, particular market, and/or or markets, to name afew. For example, the stock ID information 3802B, may be a stock symbolor ticker.

Stock price information 3802C may be information that relates to theprice of one or more of: a particular stock, stocks, particularcommodity, commodities, particular market, and/or or markets, to name afew. For example, the stock price information 3802C may indicate thatGoogle® is trading at 500 dollars a share. In embodiments, stock priceinformation 3802C may be information relating to the stock price at thetime that the stock was purchased by the user. For example, if Stock Awas purchased by user one year ago at a price of 100 dollars a share,stock price information 3802C may indicate the price of 100 dollars ashare, as opposed to the value that Stock A is currently trading at. Inembodiments, stock price information 3802C may include multiple stockprices. For example, stock price information 3802C may include the priceat which the user bought the stock and the price at which the stock iscurrently trading (e.g. purchase price and current price respectively).

Buy/sell data information 3802D may be information that relates towhether the user bought or sold the stock, commodity, or asset within amarket which was identified in Stock ID information 3802B. The buy/selldata information 3802D may be whether or not the user has purchased aspecific stock. The buy/sell data information 3802D may be whether ornot the user has sold the specific stock. In embodiments the buy/selldata information 3802D, may indicate where or through what exchange theuser purchased or sold the specific stock through. Buy/sell datainformation 3802D may refer to multiple instances of the user buyingand/or selling a specific product. In embodiments, buy/sell datainformation 3802D may be associated with one or more of the productidentified in stock ID information 3802B; quantity information 3802E,and/or timing information 3802F. For example, if Stock A is identifiedin stock ID information 3802B, the buy/sell data information 3802D mayindicate whether the user bought or sold (or neither) Stock A.

Quantity information 3802E be information that relates to whether thevolume of a stock, commodity, or asset within a market purchased and/orsold (where the value of each could be zero—i.e. the user did notpurchase or sell any stock). Quantity information 3802E, in embodiments,may have multiple entries. For example, if there were multiple instancesof a user buying and/or selling the product identified in stock IDinformation 3802B, quantity information 3802E may include the volume ofproduct sold and/or bought at each instance of the user buying and/orselling the product. In embodiments, quantity information 3802E may beassociated with one or more of the product identified in stock IDinformation 3802B; buy/sell data information 3802D, and/or timinginformation 3802F. For example, the quantity information 3802E may bethe quantity of stock (identified in stock ID information 3802A) thatwas sold (identified as sold in buy/sell data 3802D) by the user.

Timing information 3802F may be information that relates to the time atwhich a user bought or sold a product identified in stock ID information3802B. In embodiments, time may refer to one or more of the following:the time of day, the date, the month, the quarter, and/or the year. Inembodiments, timing information 3802F may be multiple times. Forexample, if the user had multiple instances identified in buy/sell datainformation 3802D, the timing information may include the time at whicheach instance occurred. In embodiments, timing information 3802F may beassociated with one or more of the product identified in stock IDinformation 3802B; quantity information 3802E, and/or buy/sell data3802D.

Timestamp information may be similar to first timestamp 3502C and secondtimestamp 3602C described above in connection with FIGS. 25A, 35, and 36, the descriptions of which applying herein.

Location information as described herein, may be similar to locationinformation 2802C and location information 2902C described above inconnection with FIGS. 22, 23, 27, and 28 , the description of whichapplying herein. As described above, location information may be usedfor determining the authenticity of the third stock market response 3802and/or determining a reliability rating of a user associated with thelaptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s) ofthe first group of user devices 2708 which transmitted the third stockmarket response 3802).

Proximity information, as described herein, may be similar to theproximity information described above in connection with FIGS. 22, 23,27, and 28 , the description of which applying herein. As describedabove, proximity information may be used for determining theauthenticity of the third stock market response 3802 and/or determininga reliability rating of a user associated with the laptop 2708-A and/orthe cellphone 2708-C (e.g. the user device(s) of the first group of userdevices 2708 which transmitted the third stock market response 3802).

Audio data, as described herein, may be similar to the audio datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, audio data maybe used for determining the authenticity of the third stock marketresponse 3802 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe third stock market response 3802).

Image data, as described herein, may be similar to the image datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, image data maybe used for determining the authenticity of the third stock marketresponse 3802 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe third stock market response 3802).

Video data, as described herein, may be similar to the video datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, video data maybe used for determining the authenticity of the third stock marketresponse 3802 and/or determining a reliability rating of a userassociated with the laptop 2708-A and/or the cellphone 2708-C (e.g. theuser device(s) of the first group of user devices 2708 which transmittedthe third stock market response 3802).

In embodiments, the computing device 2702 may use information within thethird stock market response 3802, (including e.g. user information3802A, Stock ID information 3802B, Stock price information 3802C,Buy/sell data information 3802D, Quantity information 3802E, Timinginformation 3802F, timestamp information, location information,proximity information, audio data, and/or image data, to name a few) todetermine the authenticity of the third stock market response 3802and/or the reliability of a user (and/or filter out unreliableresponses). The process of determining the reliability of a user and/orfiltering out unreliable responses may be similar to the processesdescribed above in connection with FIGS. 22, 23, 24, 27, 28, 29, and 32, the descriptions of which applying herein.

Referring to FIG. 25B, at step S2558, the computing device (e.g.computing device 2702) stores, in one or more databases, the third stockmarket response (e.g. third stock market response 3802) of each userdevice of the plurality of user devices from which the second marketresponse was received (e.g. first group of user devices 2708). in theone or more databases. In embodiments, the one or more databases, asmentioned above with respect to step S2520, may be: internal storage1808A, external storage 1808B, memory/storage 1815, system memory 1804,and/or storage 1903, to name a few. Step S2520 may be similar to stepS2212 described above in connection with FIGS. 22, 27, and 28 , thedescription of which applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe third stock market response 3802, the computing device 2702 maydetermine the authenticity of the third stock market response 3802. Theprocess of determining the authenticity of the third stock marketresponse 3802 may be similar to the process of authenticating the firstresponse 2802 of FIG. 28 and/or the first response 2902 of FIG. 29 ,described above in connection with FIGS. 22, 23, 27, 28, 29, and 30 ,the descriptions of which applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe third stock market response 3802, the computing device may determinethe reliability of the plurality of users associated with the electronicdevices (e.g. laptop 2708-A, cellphone 2708-C) that transmitted thethird stock market response 3802. The process of determining thereliability of the plurality of users associated with the electronicdevices (e.g. laptop 2708-A, cellphone 2708-C) that transmitted thethird stock market response 3802 may be similar to the process ofdetermining the reliability of the users associated with electronicdevices that transmitted the first response 2802 of FIG. 28 and/or thefirst response 2902 of FIG. 29 , described above in connection withFIGS. 22, 23, 27, 28, 29, and 30 , the descriptions of which applyingherein.

In embodiments, if the computing device 2702 has previously determinedthe reliability rating of a user, the computing device 2702 may updatethe reliability rating of the user. The updating of the reliabilityrating may be similar to the updating of the reliability ratingdescribed above in connection with the processes of FIGS. 22 and 23 ,the descriptions of which applying herein.

At step S2560, the computing device (e.g. computing device 2702)accesses at least the first stock market response (e.g. first stockmarket response 3502), the second stock market response (e.g. secondstock market response 3602), and the third stock market response (e.g.third stock market response 3802) provided by each user device of thefirst group of user devices (e.g. first group of user devices 2708). Inembodiments, the computing device 2702 may access the first stock marketresponse 3502 and/or the second stock market response 3602 by receivingthe stored first stock market response 3502, the stored second stockmarket response 3602, and/or the stored third stock market response 3802from one or more of: internal storage 1808A, external storage 1808B,memory/storage 1815, system memory 1804, and/or storage 1903, to name afew. In embodiments, the computing device 2702 may access and/or receivethe stored identification information from one or more of: internalstorage 1808A, external storage 1808B, memory/storage 1815, systemmemory 1804, and/or storage 1903, to name a few.

At step S2562, the computing device (e.g. computing device 2702)determines the trading pattern based on at least the first stock marketresponse (e.g. first stock market response 3502), the second stockmarket response (e.g. second stock market response 3602), and the thirdstock market response (e.g. third stock market response 3802) providedby the first group of user devices (e.g. first group of user devices2708), and/or the second group of user devices (associated with thesecond group mentioned herein). The trading pattern may also be based onhistorical data and/or third party information (similar to thehistorical data and third party information described above). Thecomputing device 2702, in embodiments, may use the information providedto generate a graph that maps one or more of the following pieces ofinformation: price, volume, a time associated with the price, a timeassociated with the volume, and/or a time associated with both the priceand volume, to name a few. In embodiments, the generated graph may betransmitted by the computing device 2702 to one or more of: the firstgroup of user devices 2708, the second group of user devices, and/or theplurality of user devices 3006. Based on the resulting trading pattern,the trading pattern may be analyzed by the computing device 2702.

In embodiments, the trading pattern may be analyzed by the computingdevice 2702 to determine whether a pattern has emerged that one or moreusers associated with the plurality of user devices 3006 could takeadvantage of by buying, selling, and/or holding. For example, thecomputing device 2702 may identify one or more of the followingpatterns: head and shoulders, cup and handle, double tops, doublebottoms, triangles, (e.g. symmetrical triangles, ascending triangles,and/or descending triangles, to name a few), flags and pennants, wedges,gaps, triple tops, triple bottoms, and/or saucer bottom (e.g. roundingbottom), to name a few. If an advantageous pattern is detected thecomputing device 2702 may generate and send a message to one or moreusers (e.g. users associated with the plurality of user devices 3006,users associated with the first group of user devices 2708, to name afew).

In embodiments, the computing device 2702 may determine that one or moreusers specifically can benefit from a particular action based on theinformation determined in the trading pattern. The computing device2702, in embodiments, may compare the trading pattern to availableinformation of at least one user of the plurality of users associatedwith the plurality of user devices 3006. The available information may,in embodiments, be one or more of the following: identificationinformation associated with the at least one user of the plurality ofusers; the first stock market response 3502, the second stock marketresponse 3602, and/or the third stock market response 3802. For example,if the trading pattern suggests that Stock A is about to fall in valueand Charley (a user associated with an electronic device of the firstgroup of user devices 2708) indicated that he owns a lot of Stock A, thecomputing device 2702 may generate and transmit a trading suggestionthat advises Charley to sell Stock A. The generated trading suggestionmay further include one or more of the following, a message indicatingthe trading suggestion, an image of the generated trading pattern graph,a summary of information of the trading pattern, a link to any thirdparty information used in connection with determining the tradingpattern, and/or a link to a third party that may enable the user tofollow the suggestion (e.g. a link to E-Trade®). Continuing the example,Charley may have indicated that he owns a lot of Stock A in one or moreof the following: the first stock market response 3502, the second stockmarket response 3602, the third stock market response 3802, and/oridentification information, to name a few. In embodiments, the tradingpattern may also indicate when a stock market event (e.g. Stock Afalling in price) will occur. Continuing the example, the tradingpattern may indicate that Stock A is going to fall in value in the nextquarter. Thus, the message generated and transmitted by the computingdevice 2702 to a device associated with Charley, may advise Charley tosell before the next quarter begins.

In embodiments, the trading pattern may be compared to other users sothat the computing device can generate trading suggestions based onsimilar patterns.

In embodiments, the determined trading pattern may be compared to one ormore third party trading patterns that may be received by the computingdevice 2702 from a third party source and/or a third party vendor. Oneor more third party trading patterns, in embodiments, may also be storedby the computing device 2702 in a similar manner that the abovementioned historical data and additional information from external datasources, the description of which applying herein. In embodiments, thereceived third party trading pattern may supplement, inform, and/oralter the calculated stock market prediction of the process describedabove in connection with FIG. 25A. Furthermore, in embodiments, thereceived third party trading pattern may supplement, inform, and/oralter the determined stock market prediction.

The steps of the process described in connection with FIG. 25B, inembodiments, may be rearranged or omitted.

Example 16: Method of Gathering Opinion Information

Referring now to FIG. 26 . FIG. 26 , is an illustrative flow chart of acomputer-implemented process there are illustrated flow charts of acomputer-implemented process for gathering opinion information providedby one or more users of a plurality of users of an electronic computernetwork (e.g., network 1910 in FIG. 19 ) in accordance with an exemplaryembodiment of the present invention.

The process of FIG. 26 may begin at step S2602. At step S2602, acomputing device (e.g., computing device 2702 in FIG. 27 , computer18012 in FIG. 18 , server 1901 in FIG. 19 ) may receive identificationinformation associated with each user of a plurality of users (e.g.users associated with the plurality of user devices 3006) of theelectronic computer network (e.g. network 1910). In embodiments, theidentification information may include one or more of the following: auser account associated with a user; an e-mail address associated with auser; a name associated with a user; job history data which may includeuser specific current and past employment history, biometric dataassociated with a user; gender information of the user associated withthe user device; age of the user associated with the user device;personal data of a user associated with the user device, which is eithervolunteered by the user or received via access that is consented to bythe user; past and/or present location information associated with theuser device; identification information related to a user deviceassociated with a user of the plurality of users (e.g. metadata, devicetype, etc., to name a few), electronic identification (e.g. electronicidentification card, electronic signature, etc., to name a few), and/orbiometric data of the user (e.g. audio sampling(s) of the user's voice,an image of the user, a video of the user, and/or a finger print of theuser, to name a few), to name a few. In embodiments, identificationinformation may be similar to the identification information describedabove in connection with FIGS. 22-25B, and 27-38 , the description ofwhich applying herein. Additionally, step S6402 may be similar to stepsS2202, S2302, S2402, and/or S2502 described above in connection withFIGS. 22-25A respectively, the description of which applying herein.

The computing device 2702 may determine a first group of usersassociated with a first group of user devices (e.g. first group of userdevices 2708). For example, if the computing device 2702 is going tosend a market query regarding a specific market (e.g. bonds) once theidentification information is received, the computing device 2702 maygroup users who have a job or who have had a job that is or is relatedto the specific market. In embodiments, the computing device 2702 maydetermine the first group of users in a similar manner as describedabove in connection with FIGS. 22, 23, 24, 27-38 , the description ofwhich applying herein.

At step S2604, the computing device (e.g. computing device 2702) storesin one or more databases the identification information. In embodiments,the one or more databases may be: internal storage 1808A, externalstorage 1808B, memory/storage 1815, system memory 1804, and/or storage1903, to name a few. The identification information may be stored inaccordance with the received identification information. For example,the identification indication may indicate an age of the users. In thisexample, the computing device 2702 may store the identificationinformation by age range (e.g. 18-25 ages grouped together, 26-40 agesgrouped together, etc.). As another example, the identificationinformation may be stored based on whether a user is part of a specificgroup of users. For example, identification information of the firstgroup of users associated with the first group of user devices 2708 maybe stored together. Step S2604 may be similar to step S2204 describedabove in connection with FIGS. 22, 27, and 28 , the description of whichapplying herein.

At step S2606, the computing device (e.g. computing device 2702) maytransmits market data to at least a first group of user devices (e.g.first group of user devices 2708) associated with a first group of usersof the plurality of users (e.g. users associated with the plurality ofuser devices 3006) of the electronic computer network (e.g. network1910). The purpose of the market data may be, in embodiments, to lay thegroundwork to enable a user to make a prediction related to a specificmarket, for example, Market A. Thus, in embodiments, referring to FIG.39A, the market data may include text data representing a message whichmay include one or more of: (i) past price information 3902A; (ii) pastvolume information 3902B; (iii) a first timestamp 3902C, (iv) additionalhistorical data (described above in connection with FIGS. 35-36 ); (v)third party information (described above in connection with FIGS. 35-36); (vi) stock ID information related to one or more of the past priceinformation 3902A and/or past volume information 3902B (e.g. stock IDinformation 3802B where the information is related to the market whichis the subject of the market data); (vii) audio data associated with themarket data (e.g. a podcast related to the market data); (viii) imagedata associated with the market data (e.g. one or more graphs of themarket data); and/or (ix) video data associated with the market data(e.g. a televised interview related to the market data), to name a few.

Before transmitting the market data, in embodiments, the computingdevice (e.g. computing device 2702) may generate a market data message(e.g. market data 3902). the market data message may be generated by thecomputing device 2702, by first receiving the information to be includedwith the market data message. The information to be included in themarket data message may be received by the computing device 2702 fromthird party sources and/or third party vendors. Additionally, asmentioned above with respect to historical data and third partyinformation, the market data information may be retrieved from the oneor more databases of the computing device 2702. Once the message isgenerated, in embodiments, market data 3902 may transmitted fromcomputing device 2702 to the first group of user devices 2708.

Past price information 3902A may refer to information that includes oneor more prices associated with a first specific market and/or firstspecific industry. In embodiments, the past price information 3902A mayinclude past price information for the first specific market and/orfirst specific industry over a predetermined period of time. Thepredetermined period of time may be one or more of: hours, days, weeks,months, quarters, seasons, and/or years. The computing device 2702 maydetermine the predetermined period of time based on the first specificmarket and/or first specific industry. For example, a user may makerequire price information over a period of five years to have theability to make an accurate prediction regarding future price of MarketA. Continuing the example, a user may make require price informationover a period of three years to have the ability to make an accurateprediction regarding future pricing of Market B. Thus, the computingdevice 2702 may provide past price information 3902A, including priceinformation over the past five years, for market data 3902 related toMarket A. Additionally, the computing device 2702 may provide past priceinformation 3902A, including price information over the past threeyears, for market data 3902 related to Market B. The predeterminedamount of time may also be a default amount of time. The predeterminedamount of time may also be based on the received identificationinformation. If, for example, the computing device 2702 determines thatthe first group of users associated with the first group of user device2708 is a sophisticated group of investors, the computing device 2702may generate and transmit a market data message for a sophisticatedgroup of investors. Additionally, if, for example, the computing device2702 determines that the first group of users associated with the firstgroup of user device 2708 is an unsophisticated group of investors, thecomputing device 2702 may generate and transmit a market data messagefor a group of unsophisticated groups of investors.

Past volume information 3902B may refer to information that includes oneor more volumes associated with a second specific market and/or secondspecific industry. In embodiments, the past volume information 3902B mayinclude past pricing information for the second specific market and/orsecond specific industry over a predetermined period of time. Thepredetermined amount of time may be the same predetermined amount oftime associated with past price information 3902A. For example, themarket data 3902 may provide past price information 3902A and pastvolume information 3902B in a generated graph, the graph showing pastprices and volumes over the predetermined amount of time. Inembodiments, the past volume information 3902B may be over a differentpredetermined amount of time (which may be determined in a similarmanner to the past price information 3902A).

In embodiments, the first specific market and/or first specific industrymay be the same as the second specific market and/or second specificindustry.

Past price information 3902A and past volume information 3902B, may besimilar to the past financial market conditions described above inconnection with the first market query 3102 of FIG. 31 , the descriptionof which applying herein.

First timestamp 3902C may refer to a timestamp associated with the pastprice information 3902A and/or past volume information 3902B. Forexample, the first timestamp 3902C may include the aforementioned (withrespect to the past price information 3902A and/or past volumeinformation 3902B) predetermined amount of time. In embodiments, firsttimestamp 3902C may be similar to first timestamp 3502C, secondtimestamp 3602C, first timestamp 3202C, second timestamp 3302C,timestamp 2802D, and timestamp 2902D described above in connection withFIGS. 22-35 , the descriptions of which applying herein. In embodiments,the market data comprises: (i) past price information; (ii) past volumeinformation; and (iii) a first timestamp.

In embodiments, the market data 3902 may include executable machinereadable instructions that allow for the computing device 2702 todetermine when one or more user devices of the first group of userdevices 2708 received, opened, and/or began to respond to the marketdata 3902. For example, the market data 3902 may include MessageDisposition Notification(s) (MDN). In embodiments, if the user neveropens (thus did not read or begin to respond to the market data 3902)the market data 3902. The computing device 2702 may determine via theexecutable machine instructions. If, for example, the user never openedthe market data 3902, but sent the first market response 4002 and/or thesecond market response 4102, the computing device 2702 may determinethat the information received in the first market response 4002 and/orthe second market response 4102 is not authentic. If, for example, theuser never opened the market data 3902, but sent the first marketresponse 4002 and/or the second market response 4102, the computingdevice 2702 may determine that the user associated with the device thatsent the first market response 4002 and/or the second market response4102 is unreliable or has a higher probability of being unreliable (e.g.the first market response may include the user(s) impressions of themarket data and/or answers to queries regarding the market data).

In embodiments, the market data 3902 may be generated in response to thecomputing device 2702 determining that an event is occurring that isrelated to a market. For example, if there is a report of a first marketdropping in value, the computing device 2702 may generate market data3902 to give users context to determine a prediction regarding the firstmarket's future performance. The market data 3902 may be specific to theevent related to the market. For example, market data 3902 may includetext that states, “we have received a report that the First Market isdropping in value, here is some relevant information regarding the FirstMarket.” Additionally, in embodiments, the market data 3902 may providea link to the report and/or an excerpt of the report. Furthermore, inembodiments, the market data 3902 may also include user specificinformation. For example, the market data 3902, may state “John, we havereceived a report that the First Market is dropping in value, here issome relevant information regarding the First Market.”

In embodiments, the computing device 2702 may determine and/or storemarket data start information. Market data start information, inembodiments, may refer to the time at which an event related to themarket data started occurring.

The market data message (e.g. market data 3902) may include machinereadable instructions to present a message on the one or more userdevices of the first group of user devices 2708. In embodiments, themessage may be related to the market data mentioned above.

At step S2608, the computing device (e.g. computing device 2702)generates a first market query (e.g. first market query 3904) related tothe market data (e.g. market data 3902). The purpose of the first marketquery 3904, in embodiments, may be to determine one or more impressionsof one or more users receiving the market data. In embodiments, thefirst market query 3904, may be related to market data 3902. Forexample, referring to FIG. 39B, the first market query 3104 may includea message stating “Does the market data you received give you animpression as to how the market is performing?”

In embodiments, the first market query 3904 may include executablemachine readable instructions that allow for the computing device 2702to determine when one or more user devices of the first group of userdevices 2708 received, opened, and/or began to respond to the firstmarket query 3904. For example, the first market query 3904 may includeMessage Disposition Notification(s) (MDN).

In embodiments, the first market query 3904 may be generated in responseto the computing device 2702 generating and/or transmitting the marketdata 3902. In embodiments, the first market query 3904 may be generatedin response to the computing device 2702 determining that an event isoccurring that is related to a market. For example, if there is a reportof a first market dropping in value, the computing device 2702 maygenerate the first market query 3904 to give users context to determinea prediction regarding the first market's future performance. The firstmarket query 3904 may be specific to the event related to the market.For example, first market query 3904 may include text that states, “wehave received a report that the First Market is dropping in value, afterviewing the market data we sent you, do you have any impressions withregards to the First Market?” Additionally, in embodiments, the firstmarket query 3904 may provide a link to the report and/or an excerpt ofthe report. Furthermore, in embodiments, the first market query 3904 mayalso include user specific information. For example, the first marketquery 3904, may state “Jason, we have received a report that the FirstMarket is dropping in value, after viewing the market data we sent you,do you have any impressions with regards to the First Market?”

In embodiments, the first market query 3902 may include a questionregarding information that the computing device 2702 has alreadyconfirmed. For example, the computing device 2902 may have alreadyconfirmed that the First Market dropped by 5 points last quarter. Inthis example, the first market query 3902 may state “Bob, how did theFirst Market perform last quarter? Additionally, do you have anyimpressions with regards to the First Market's performance?” The firstquestion, “how did the First Market perform last quarter?” may be askedto confirm the reliability of the user. The reliability of the user, inthis case, may be related to whether the user reviewed the market data3902. The remaining question, “do you have any impressions with regardsto the First Market's performance?” may be asked to query theimpressions of the user with regards to the First Market performance. Inembodiments, the computing device 2702 may only review answers of thesecond question from users that provide the correct answer to the firstquestion.

In embodiments, the computing device 2702 may determine and/or storemarket start information. Market start information, in embodiments, mayrefer to the time at which one or more events related to the market data3902 started occurring.

The first market query (e.g. first market query 3904) may includemachine readable instructions to present an inquiry message on the oneor more user devices of the first group of user devices 2708. Inembodiments, the inquiry message may be related to the market datamentioned above. In embodiments the inquiry message may be related tomarket data that was modified or did not take place. For example, thecomputing device 2702 may have already confirmed that the First Marketdropped by 3 points last quarter. In this example, the computing device2702 may generate a first market query that may state “Do you know howthe First Market performed last quarter?” If the user respondscorrectly, then the user may be determined as a reliable source. If theuser responds incorrectly, then the user may be determined as anunreliable source. As another example, the first market query 3904 mayinclude an inquiry message that queries the user about informationregarding the subject to of the market data 3902 that has changed sincemarket data 3902 was sent to the first group of user devices 2708. Ifthe user responds correctly to the inquiry, then the user may bedetermined as a reliable source. If the user responds incorrectly to theinquiry, then the user may be determined as an unreliable source. As yetanother example, the first market query 3904 may include an inquirymessage that queries the user about information regarding the subject toof the market data 3902 that was incorrect. The market data 3902 mayhave been purposely incorrect to check the reliability of one or moreusers or the market data 3902 may have included information that hassince been found to be incorrect. If the user responds correctly to theinquiry, then the user may be determined as a reliable source. If theuser responds incorrectly to the inquiry, then the user may bedetermined as an unreliable source.

At step S2610, the computing device transmits the first market query toone or more user devices of the first group of user devices. Thecomputing device (e.g. computing device 2702) transmits the first marketquery (e.g. first market query 3904) to at least a first group of userdevices (e.g. first group of user devices 2708) associated with a firstgroup of users of the plurality of users (e.g. users associated withplurality of user devices 3006) of the electronic computer network (e.g.network 1910). Referring to FIG. 39B, computing device 2702, maytransmit the first market query 3904 to the first group of user devices2708. In embodiments, the first market query 3904 may be transmittedover network 1910. In embodiments, the first group of user devices 2708may include one or more of: laptop 2708-A, wearable device 2708-B,and/or cell phone 2708-C. First group of user devices 2708, inembodiments, may be associated with a first group of users of theplurality of users (e.g. plurality of users 3006 described in connectionwith FIG. 30 ). The step S2610 may be similar to step S2208 describedabove in connection with FIGS. 22 and 27 , the description of whichapplying herein.

Referring back to FIG. 26 , at step S2612, the computing device mayreceive a first market response (e.g. first market response 4002). Thefirst market response, in embodiments, may be received from one or moreuser devices of the first group of user devices (e.g. first group ofuser devices 2708). Referring to FIG. 40 , in embodiments, the computingdevice 2702 may receive the first market response 3202 from a first userassociated with the laptop 2708-A and a second user associated with thecellphone 2708-C. The first market response 4002, in embodiments, mayinclude multiple responses specific to each user. For example, the firstmarket response 4002 may include a specific first user market responseand a specific second user market response. While not shown in FIG. 40 ,continuing the example, the first user market response may be receivedby the computing device 2702 at a different time than the time at whichthe computing device 2702 receives the second user market response.While it is clear in the art that messages from different electronicdevices do not need to be received at the same time, for brevity andclarity purposes, only one first market response (First Market Response4002) is shown in FIG. 40 from at least one user device of the firstgroup of user devices. In embodiments, the first market responsecomprises: (i) user identification information unique to the respectiveuser device and associated with the respective user associated with therespective user device; (ii) impression sentiment information related tothe respective user's impression of the market data; and (iii) a secondtimestamp.

In embodiments, the first market response 3202 may include one or moreof the following: (i) user information 4002A unique to the respectiveuser associated with the respective user device providing the firstmarket response; (ii) impression sentiment information 4002B related tothe respective user's impression of the market data; (iii) a secondtimestamp 4002C; (iv) location information associated with a location ofthe respective user device associated with the respective user; (v)proximity information; (vi) audio data associated with the impressionsof the user; (vii) image data associated with the impressions of theuser; and/or (viii) video data associated with the impressions of theuser, to name a few.

User information 4002A may be similar to user information 2802A and userinformation 2902A described above in connection with FIGS. 22, 23, 27,and 28 , the description of which applying herein. User information4002A, in embodiments, may also include one or more of the following:job history data which may include user specific current and pastemployment history (similar to the job history data described above inconnection with the identification information received in step S2602),a user account associated with a user; an e-mail address associated witha user; a name associated with a user; biometric data associated with auser; gender information of the user associated with the user device;age of the user associated with the user device; personal data of a userassociated with the user device, which is either volunteered by the useror received via access that is consented to by the user; locationinformation associated with the user device; identification informationrelated to a user device associated with a user of the plurality ofusers (e.g. metadata, device type, etc., to name a few), and/orelectronic identification (e.g. electronic identification card,electronic signature, etc., to name a few), to name a few. Furthermore,as described above in connection with FIGS. 22, 23 27, and 28, userinformation 4002A may include connection information. Connectioninformation, as described above, may enable the computing device 2702 toaccurately set an upper and/or lower time lag limit which may be usedfor authentication of the first market response 4002 and/or adetermining a reliability rating of a user associated with the laptop2708-A and/or the reliability rating of a user associated with thecellphone 2708-C (e.g. the user devices of the first group of userdevices 2708 which sent the first market response 4002).

Impression sentiment information 4002B may be information received froma user device (e.g. cellphone 2708-C) that is responsive to one or moreof the market data 3902 and/or the first market query 3904. For example,impression sentiment information 4002B may include information thatstates the thoughts of the user with regards to the performance of themarket (the market being the subject of one or more of: the market data3902 and/or the first market query 3904). In embodiments, impressionsentiment information 4002B may include audio, image, and/or video data,allowing the user to more fully express his or her impression. Inembodiments, continuing the examples mentioned above, the first marketresponse 4002 may be responsive to a query in which the computing device2702 may have already confirmed. As with the example above, thecomputing device 2702 may have confirmed that the First Market droppedby 5 points last quarter. In this example, the first market query 3904may state “Bob, how did the First Market perform last quarter?Additionally, do you have any impressions with regards to the FirstMarket's performance?” The first question, “how did the First Marketperform last quarter?” may have been asked to confirm the reliability ofthe user. The remaining question, “do you have any impressions withregards to the First Market's performance?” may be to illicit a responsein which the user will give his or her impressions. The first marketresponse 3202 from the laptop 2708-A and/or the cellphone 2708-C, mayinclude information that is received via message (e.g. “Yes, the FirstMarket dropped by five points last quarter. The First Market is doingfine, a little bump in the road.”) and/or information that is receivedvia answers to a prompt (e.g. a prompt that asks “How did the FirstMarket perform last quarter”—where the prompt has a few preselectedanswers—“(a) Up 5 Points; (b) Down 5 Points; (c) I don't know” and theuser selects one or more of the options presented).

Second timestamp 4002C may be similar to first timestamp 3902C, firsttimestamp 3502C, second timestamp 3602C, first timestamp 3202C, secondtimestamp 3302C, timestamp 2802D, and timestamp 2902D described above inconnection with FIGS. 22-39 , the descriptions of which applying herein.As described above, second timestamp 4002C may enable the computingdevice 2702 to calculate a time lag associated with the first marketresponse 4002, the market data 3902, and/or the first market query 3904.To compute the time lag, the second timestamp 4002C may include multipletime stamps (e.g. a time at which the first market query 3904 was sent,a time at which the first market query 3904 was opened, a time at whichthe first market response 4002 was started, a time at which the firstmarket response 4002 was transmitted, and/or a time at which the firstmarket response 4002 was received, to name a few). The multiple timeswithin the second timestamp 4002C may be used to compute the time lag.For example, the computing device 2702 may determine a time lag bydetermining the amount of time between a first time when the firstmarket query 3904 was opened by the laptop 2708-A and a second time whenthe first market response 4002 was transmitted by the laptop 2708-A tothe computing device 2702. As another example, the computed time lag maybe the time difference between a first time associated with the firsttimestamp 3902C (e.g. a time at which the market data 3902 was sent, atime at which the market data 3902 was opened, to name a few) and asecond time associated with the second timestamp 4002C. As anotherexample, in embodiments, the computed time lag may be the timedifference between a first time which is not associated with the secondtimestamp 4002C (e.g. a time of a particular market event element of themarket data 3902) and a second time which is associated with the secondtimestamp 4002C (e.g. the time at which the first market response 4002was received by the computing device 2702). The computed time lag, inembodiments and as mentioned above, may be compared to upper and/orlower time lag limits. The comparison of the computed time lag to theupper and/or lower time lag limits, as mentioned above, may be used forauthentication of the first market response 4002 and/or a determining areliability rating of a user associated with the laptop 2708-A and/orthe cellphone 2708-C (e.g. the user device(s) of the first group of userdevices 2708 which sent the first market response 4002). Moreover, asmentioned above, the upper and/or lower time lag limits (thresholds) maybe predetermined. The predetermined upper and/or lower time lag limitsmay be determined in the context of the previously mentioned connectioninformation (e.g. if the connection is poor, the lower and/or upperlimits may be increased to account for the poor connection, if theconnection is good, the lower and/or upper limits may be decreased toaccount for the good connection, etc.).

Location information as described herein, may be similar to locationinformation 2802C and location information 2902C described above inconnection with FIGS. 22, 23, 27, and 28 , the description of whichapplying herein. As described above, location information may be usedfor determining the authenticity of the first market response 4002and/or determining a reliability rating of a user associated with thelaptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s) ofthe first group of user devices 2708 which transmitted the first marketresponse 4002).

Proximity information, as described herein, may be similar to theproximity information described above in connection with FIGS. 22, 23,27, and 28 , the description of which applying herein. As describedabove, proximity information may be used for determining theauthenticity of the first market response 4002 and/or determining areliability rating of a user associated with the laptop 2708-A and/orthe cellphone 2708-C (e.g. the user device(s) of the first group of userdevices 2708 which transmitted the first market response 4002).

Audio data, as described herein, may be similar to the audio datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, audio data maybe used for determining the authenticity of the first market response4002 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the firstmarket response 4002).

Image data, as described herein, may be similar to the image datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, image data maybe used for determining the authenticity of the first market response4002 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the firstmarket response 4002).

Video data, as described herein, may be similar to the video datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, video data maybe used for determining the authenticity of the first market response4002 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the firstmarket response 4002).

In embodiments, the computing device 2702 may use information withinfirst market response 4002, (including e.g. user information 4002A,impression sentiment information 4002B, second timestamp 4002C, locationinformation, proximity information, audio data, and/or image data, toname a few) to determine reliability and/or filter out unreliableresponses. For example, the computing device 2702 may receive theimpression sentiment information 4002B of first market response 4002 andthe second timestamp 4002C of first market response 4002. In thisexample, the second timestamp 4002C may include the began drafting firstmarket response time of the first market query 3904 and the sent time ofthe first market response 4000. In embodiments, the computing device2702 may compare the began drafting first market response time and thesent time of the first market response to calculate a time lag. Thistime lag may be used to determine the reliability of the user associatedwith a user device that transmitted the first market response 4002. Inembodiments, this time lag may be viewed in the context of theimpression sentiment information 4002B. If, for example, the time lag ishigh, and the amount of information contained within the impressionsentiment information 4002B is high, the computing device 2702 maydetermine that the user associated with the first market response 4002is reliable and/or has a higher probability of being reliable becausethe high lag time may be due to the amount of information input by theuser. If, for example, the time lag is high and the amount ofinformation contained within the impression sentiment information 4002Bis low, the computing device 2702 may determine that the first marketresponse 4002 is unreliable and/or has a higher probability of beingunreliable because the amount of content sent by the user does notreflect the amount of time spent crafting the first market response4002. If, for example, the time lag is low and the amount of informationcontained within the impression sentiment information 4002B is high, thecomputing device 2702 may determine that the first market response 4002is unreliable and/or has a higher probability of being unreliablebecause a user may have not been able to send a response with the highamount of information within the amount of time.

In embodiments, the computing device computing device 2702 may store, inthe one or more databases, the first market response 4002 of each userdevice of the plurality of user devices of the first group of userdevices (e.g. first group of user devices 2708) from which the firstmarket response 4002 was received. In embodiments, the one or moredatabases, as mentioned above, may be: internal storage 1808A, externalstorage 1808B, memory/storage 1815, system memory 1804, and/or storage1903, to name a few.

In embodiments, once the computing device 2702 receives and/or storesthe first market response 4002, the computing device 2702 may determinethe authenticity of the first market response 4002. The process ofdetermining the authenticity of the first market response 4002 may besimilar to the process of authenticating the first response 2802 of FIG.28 and/or the first response 2902 of FIG. 29 , described above inconnection with FIGS. 22, 23, 27, 28, 29, and 30 , the descriptions ofwhich applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first market response 4002, the computing device 2702 may determinethe reliability of the plurality of users associated with the electronicdevices (e.g. laptop 2708-A, cellphone 2708-C) that transmitted thefirst market response 4002. The process of determining the reliabilityof the plurality of users associated with the electronic devices (e.g.laptop 2708-A, cellphone 2708-C) that transmitted the first marketresponse 4002 may be similar to the process of determining thereliability of the users associated with electronic devices thattransmitted the first response 2802 of FIG. 28 and/or the first response2902 of FIG. 29 , described above in connection with FIGS. 22, 23, 27,28, 29, and 30 , the descriptions of which applying herein.

Referring to FIG. 26 , at step S2614, the computing device (e.g.computing device 2702) may generates a second market query (e.g. secondmarket query 3906) related to future market parameters. The purpose ofthe second market query, in embodiments, may be to receive predictions(with regards to the market of the market data 3902) from a plurality ofusers. In embodiments, the second market query may be related toeliciting predictions from one or more users with regards to pricesassociated with the market of the market data 3902 and/or volumeassociated with the market of the market data 3902. For example,referring to FIG. 39B, the second market query 3906 may include amessage stating:

“Do you have any thoughts with regards to how the market will perform?

Price: ______

Volume: ______”

In embodiments, the second market query 3906 may include a queryrelating to future market conditions of the market that was the subjectof the one of the market data 3902 and/or the first market query 3904.Thus, the computing device 2702 may use the first market query 3904 todetermine whether a user is reliable in regard to information withinmarket data 3902, then use the second market query 3906 to ask for aprediction with regards to the specific market, the predictions, inembodiments, being a price prediction and/or a volume prediction. Inembodiments, second market query 4002 may include predictions, which mayinclude one or more of: price prediction, volume prediction, futureplans of purchasing, timing of the future plans of purchasing, futuremarket predictions and/or reasons regarding the decisions and/orpredictions related to the specific market, to name a few. Priceprediction may refer to a price that the user believes the market willbe worth and/or trading at. Volume prediction may refer to the futurevolume of purchasers, sellers, and/or owners within the market. Thefuture plans of purchasing may refer to whether the user will purchase afirst specific asset. Timing of the future plans of purchasing may referto when the user is planning on purchasing a second specific asset.Future market predictions may refer to a prediction of how a thirdspecific asset will perform over a certain period of time. Inembodiments, the first specific asset, the second specific asset, and/orthe third specific asset may be the same asset. In embodiments, thereasons regarding the decisions and/or predictions related to the futuremarket conditions may refer to a query that allows the user to give hisor her reasons for their predictions.

In embodiments, the second market query 3906 may include executablemachine readable instructions that allow for the computing device 2702to determine when one or more user devices of the first group of userdevices 2708 received, opened, and/or began to respond to the secondmarket query 3906. For example, the second market query 3906 may includeMessage Disposition Notification(s) (MDN).

In embodiments, the second market query 3906 may be generated inresponse to the computing device 2702 determining that an event (e.g. amarket event) is occurring that is related to a market. For example, ifthere is a report of a First Market increasing in value, the computingdevice 2702 may generate the second market query 3906 to determine ifreliable users associated with the first group of user devices 2708believes the First Market will continue to increase in value. The secondmarket query 3906 may be specific to the market event related to themarket. For example, second market query 3906 may include text thatstates, “we have received a report that the First Market is increasingin value, do you believe the First Market will continue to performwell?” Additionally, in embodiments, the second market query 3906 mayprovide a link to the report and/or an excerpt of the report.Furthermore, in embodiments, the second market query 3906 may alsoinclude user specific information. For example, the second market query3906, may state “Jennifer, we have received a report that the FirstMarket is increasing in value, as someone who works in the First Market,do you believe the First Market will continue to perform well?” Inembodiments, the second market query 3906 may include a questionregarding information that the computing device 2702 has alreadyconfirmed, similar to the first market query 3906 described above, thesame description applying herein.

In embodiments, the computing device 2702 may determine and/or storemarket start information. Market start information, in embodiments, mayrefer to the time at which the event (the event which may be the subjectof the process described in FIG. 26 ) started occurring.

The second market query (e.g. second market query 3906) may includemachine readable instructions to present an inquiry message on the oneor more user devices of the first group of user devices 2708. Inembodiments, the inquiry message may be related to the future financialmarket condition(s) mentioned above.

In embodiments the inquiry message of the second market query 3906 maybe related to financial market condition(s) that were modified or didnot take place. This may be similar to the conditions that were modifiedor did not take place that may be sent with the first market query 3904,the description of which applying herein.

At step S2616, the computing device transmits the second market query toone or more user devices of the first group of user devices. Referringback to FIG. 26 , at step S2616, the computing device (e.g. computingdevice 2702) transmits the second market query (e.g. second market query3906) via a network (e.g. network 1910). In embodiments, the secondmarket query may be transmitted to the first group of devices (e.g.first group of devices 2708). In embodiments, the second market querymay be transmitted to a second group of user devices. The second groupof user devices, in embodiments, may be user devices associated with oneor more of the following: authenticated users who transmitted the firstmarket responses 4002 (the authentication process, in embodiments, beingperformed by the computing device 2702B), and/or users who, via anassociated user device, transmitted the first market responses 4002where the computing device 2702 has given the users a reliability ratingof RELIABLE (or given a reliability rating which exceeds a predeterminedthreshold). In embodiments, the second market query 3906 may betransmitted to a plurality of user devices (e.g. plurality of userdevices 3006) associated with the plurality of users of an electroniccomputer network (e.g. network 1910).

Referring to FIG. 39B, computing device 2702, may transmit the secondmarket query 3906 to the first group of user devices 2708. Inembodiments, the second market query 3906 may be transmitted overnetwork 1910. In embodiments, the first group of user devices 2708 mayinclude one or more of: laptop 2708-A, wearable device 2708-B, and/orcell phone 2708 C. First group of user devices 2708, in embodiments, maybe associated with a first group of users of the plurality of users(e.g. plurality of users associated with the plurality of user devices3006 described in connection with FIG. 30 ). The step S2616 may besimilar to step S2208 described above in connection with FIGS. 22 and 27, the description of which applying herein.

Referring back to FIG. 26 , at step S2618, the computing device (e.g.computing device 2702) may receive a second market response (e.g. secondmarket response 4102). The second market response, in embodiments, maybe received from one or more user devices of the first group of userdevices (e.g. first group of user devices 2708). In embodiments, thesecond market response may be received from one or more user devices ofthe aforementioned (with regards to the process of FIG. 24 ) secondgroup of user devices. Referring to FIG. 41 , the computing device 2702may receive the second market response 4102 from a first user associatedwith the laptop 2708-A and a second user associated with the cellphone2708-C. The second market response 4102, in embodiments, may includemultiple responses specific to each user. For example, the second marketresponse 4102 may include a specific first user market response and aspecific second user market response. While not shown in FIG. 41 ,continuing the example, the first user market response may be receivedby the computing device 2702 at a different time than the time at whichthe computing device 2702 receives the second user market response.While it is clear in the art that messages from different electronicdevices do not need to be received at the same time, for brevity andclarity purposes, only one second market response (second marketresponse 4102) is shown in FIG. 41 .

In embodiments, second market response 4102 may include one or more ofthe following: (i) user identification information 4102A unique to therespective user associated with the respective user device; (ii)prediction information 4102B related to at least one of future priceinformation and future volume information; (iii) a third timestamp4102C; (iv) location information associated with a location of therespective user device associated with the respective user; (v)proximity information; (vi) audio data associated with the predictioninformation 4102B and/or observations of the user; (vii) image dataassociated with the prediction information 4102B and/or observations ofa user; and/or (viii) video data associated with the predictioninformation 4102B and/or observations of a user, to name a few.

User identification information 4102A may be similar to user information4002A, user information 3802A, user information 3602A, user information3502A, user information 3302A, user information 3202A, user information2802A, and user information 2902A described above in connection withFIGS. 22-40 , the descriptions of which applying herein.

Prediction information 4102B may be information received from a userdevice (e.g. laptop 2708-A) that is responsive to the second marketquery 3906. For example, prediction information 4102B may be responsiveto a query related to a future price and/or future volume that thecomputing device 2702 is attempting to predict. For example, in responseto a market query that states “John, after reviewing the First Marketdata and detailing your impressions, do you have a prediction of theprice and volume of the First Market?” the prediction information 4102Bmay include information that answers the question of “do you have aprediction of the price and volume of the First Market” For example, theprediction information 4102B may include text data representing amessage that states: “Yes, the First Market will be trading at 100dollars a share and have a volume of 5,000 shareholders by the end ofthe first quarter.” As another example, as shown in FIG. 39B, the secondmarket query 3906 may state:

“Do you have any thoughts with regards to how the market will perform?

-   -   Price: ______    -   Volume: ______”

In response to the second market query 3906, prediction information4102B may include text data representing a message that states: fillsout the “blanks.” For example, a user may input the following as aresponse:

“Price: $100 Volume: 5,000.”

In embodiments, the user may input the data into blanks provided in thesecond market query 3906. To send the response from the user deviceassociated with the user responding (e.g. laptop 2708-A) to thecomputing device 2702, in embodiments, the user may simply press “send”on a touch screen. In embodiments, if all of the blanks are not filledout (e.g. the user only fills out price, but not volume), theapplication associated with the interface on the user device (e.g.application 1905), may prevent the user from clicking send (e.g. givingan “incomplete form” message, stating that only complete forms may besubmitted as responses). In embodiments, if all of the blanks are notfilled out, the application 1905 may allow the second market response4102 to be transmitted from a user device to the computing device 2702.In embodiments, incomplete forms may be flagged by the computing device2702. The flagging, in embodiments, may cause the computing device 2702to do a more substantive authenticity and/or reliability ratingdetermination. This is because an incomplete form, in embodiments, mayindicate that the user is unauthentic and/or unreliable. However, inembodiments, a user may be transmitting an incomplete form simplybecause the user does not have a prediction with regards to either priceand/or volume. In these embodiments, even if the user is more strictlyscrutinized for authenticity and/or reliability ratings, the user'sincomplete prediction may demonstrate that the user is reliable becausethe user is only willing to make a prediction if he or she iscomfortable with the prediction. In embodiments, in order to gather moreinformation relating to potential price and/or volume predictionsassociated with the market subject of the second market query 3906, thequery and/or prediction information 4102B may include informationregarding timing, prices, volume, and/or reasoning, to name a few.

Third timestamp 4102C may be similar to second timestamp 4002C, firsttimestamp 3902C, first timestamp 3502C, second timestamp 3602C, firsttimestamp 3202C, second timestamp 3302C, timestamp 2802D, and timestamp2902D described above in connection with FIGS. 22-40 , the descriptionsof which applying herein. As described above, the third timestamp 4102Cmay enable the computing device 2702 to calculate a time lag associatedwith the second market response 4102, the first market response 4002,market data 3902, the first market query 3904, and/or the second marketquery 3906. To compute the time lag, the third timestamp 4102C mayinclude multiple time stamps (e.g. a time at which the first marketquery 3904 was sent, a time at which the second market query 3906 wasopened, a time at which the second market response 4102 was started, atime at which the second market response 4102 was transmitted, and/or atime at which the second market response 4102 was received, to name afew). The multiple times within the third timestamp 4102C may be used tocompute the time lag. For example, the computing device 2702 maydetermine a time lag by determining the amount of time between a firsttime when the second market query 3906 was opened by the laptop 2708-Aand a second time when the second market response 4102 was transmittedby the laptop 2708-A to the computing device 2702. As another example,the computed time lag may be the time difference between a first timeassociated with the first timestamp 3902C (e.g. a time at which themarket data 3902 was sent, a time at which the market data 3902 wasopened, to name a few) and a second time associated with the thirdtimestamp 4102C. As another example, the computed time lag may be thetime difference between a first time associated with the secondtimestamp 4002C (e.g. a time at which the first market response 4002 wassent, a time at which the first market response 4002 was opened, to namea few) and a second time associated with the third timestamp 4102C. Asyet another example, in embodiments, the computed time lag may be thetime difference between a first time which is not associated with thethird timestamp 4102C (e.g. a time of a particular market event elementof the market data 3902) and a second time which is associated with thethird timestamp 4102C (e.g. the time at which the second market response4102 was received by the computing device 2702). The computed time lag,in embodiments and as mentioned above, may be compared to upper and/orlower time lag limits (e.g. thresholds). The comparison of the computedtime lag to the upper and/or lower time lag limits, as mentioned above,may be used for authentication of the second market response 4102 and/ora determining a reliability rating of a user associated with the laptop2708-A and/or the cellphone 2708-C (e.g. the user device(s) of the firstgroup of user devices 2708 which sent the second market response 4102).Moreover, as mentioned above, the upper and/or lower time lag limits(thresholds) may be predetermined. The predetermined upper and/or lowertime lag limits may be determined in the context of the previouslymentioned connection information (e.g. if the connection is poor, thelower and/or upper limits may be increased to account for the poorconnection, if the connection is good, the lower and/or upper limits maybe decreased to account for the good connection, etc.).

Location information as described herein, may be similar to locationinformation 2802C and location information 2902C described above inconnection with FIGS. 22, 23, 27, and 28 , the description of whichapplying herein. As described above, location information may be usedfor determining the authenticity of the second market response 4102and/or determining a reliability rating of a user associated with thelaptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s) ofthe first group of user devices 2708 which transmitted the second marketresponse 4102).

Proximity information, as described herein, may be similar to theproximity information described above in connection with FIGS. 22, 23,27, and 28 , the description of which applying herein. As describedabove, proximity information may be used for determining theauthenticity of the second market response 4102 and/or determining areliability rating of a user associated with the laptop 2708-A and/orthe cellphone 2708-C (e.g. the user device(s) of the first group of userdevices 2708 which transmitted the second market response 4102).

Audio data, as described herein, may be similar to the audio datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, audio data maybe used for determining the authenticity of the second market response4102 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the secondmarket response 4102).

Image data, as described herein, may be similar to the image datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, image data maybe used for determining the authenticity of the second market response4102 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the secondmarket response 4102).

Video data, as described herein, may be similar to the video datadescribed above in connection with FIGS. 22, 23, 27, and 28 , thedescription of which applying herein. As described above, video data maybe used for determining the authenticity of the second market response4102 and/or determining a reliability rating of a user associated withthe laptop 2708-A and/or the cellphone 2708-C (e.g. the user device(s)of the first group of user devices 2708 which transmitted the secondmarket response 4102).

In embodiments, the computing device 2702 may use information withinsecond market response 4102, (including e.g. user identificationinformation 4102A, prediction information 4102B, third timestamp 4102C,location information, proximity information, audio data, and/or imagedata, to name a few) to determine the reliability rating of a userand/or filter out unreliable responses. The process of determining thereliability rating of a user and/or filtering out unreliable responsesmay be similar to the processes described above in connection with FIGS.22, 23, 24, 27, 28, 29, and 32 , the descriptions of which applyingherein. from at least one respective user device of the first group ofuser devices. In embodiments, the second market response comprises: (i)user identification information unique to the respective user associatedwith the respective user device; (ii) prediction information related toat least one of future price information and future volume information;and (iii) a third timestamp.

Referring back to FIG. 26 , at step S2620, the computing device (e.g.computing device 2702) stores the second market response (e.g. secondmarket response 4102) of each user device of the plurality of userdevices from which the second market response was received (e.g. firstgroup of user devices 2708). in the one or more databases. Inembodiments, the one or more databases, as mentioned above, may be:internal storage 1808A, external storage 1808B, memory/storage 1815,system memory 1804, and/or storage 1903, to name a few. Step S2620 maybe similar to step S2212 described above in connection with FIGS. 22,27, and 28 , the description of which applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first market response 4002 and/or the second market response 4102,the computing device 2702 may determine the authenticity of the firstmarket response 4002 and/or the second market response 4102. The processof determining the authenticity of the first market response 4002 and/orthe second market response 4102 may be similar to the process ofauthenticating the first response 2802 of FIG. 28 and/or the firstresponse 2902 of FIG. 29 , described above in connection with FIGS. 22,23, 27, 28, 29, and 30 , the descriptions of which applying herein.

In embodiments, once the computing device 2702 receives and/or storesthe first market response 4002 and/or the second market response 4102,the computing device may determine the reliability of the plurality ofusers associated with the electronic devices (e.g. laptop 2708-A,cellphone 2708-C) that transmitted the first market response 4002 and/orthe second market response 4102. The process of determining thereliability of the plurality of users associated with the electronicdevices (e.g. laptop 2708-A, cellphone 2708-C) that transmitted thefirst market response 4002 and/or the second market response 4102 may besimilar to the process of determining the reliability of the usersassociated with electronic devices that transmitted the first response2802 of FIG. 28 and/or the first response 2902 of FIG. 29 , describedabove in connection with FIGS. 22, 23, 27, 28, 29, and 30 , thedescriptions of which applying herein.

In embodiments, if the second market response 4102 indicates that theuser associated with the user device that transmitted the second marketresponse 4102 made a price and/or volume prediction, the computingdevice 2702 may use that information to determine the reliability ofthat user for future market queries (e.g. a third market query, a fourthmarket query, . . . , an N market query). For example, the computingdevice 2702 may use third party sources to determine whether thepredictions received from users turned out to be correct, or within acertain predetermined factor of error (the predetermined factor oferror, in embodiments, depending on the type of prediction made, thedifficulty of the prediction, the percentage of users who wereincorrect, and/or the percentage of users who were correct, to name afew). In embodiments, the computing device 2702 may store “correct”and/or “incorrect” predictions associated with each user. Inembodiments, the stored correct and/or incorrect predictions may beaccessed and/or used by the computing device to update and/or determinea reliability rating associated with the user who made the prediction.In embodiments, if the user made a “correct” (or correct within apredetermined factor of error), the computing device 2702 may generateand transmit a message, the message indicating the user made a correctprediction. In embodiments, if the user made a “incorrect”, thecomputing device 2702 may generate and transmit a message, the messageindicating the user made an incorrect prediction. The message (eithercorrect message or incorrect message or both), in embodiments, may alsolist past stored “correct” and/or “incorrect” predictions, allowing theuser to see how well they are predicting markets. The past storedcorrect and/or incorrect predictions may be accessed and/or used by thecomputing device 2702 to give a user a prediction score and/or aprediction grade. The score and/or grade may be based on one or more ofthe following: the amount of correct predictions as compared to theoverall number of predictions made by the user, the amount of incorrectpredictions as compared to the overall number of predictions made by theuser, an amount of correct predictions as compared to the amount ofincorrect predictions made by the user, the type of prediction made, thedifficulty of the prediction, the percentage of users who wereincorrect, and/or the percentage of users who were correct, to name afew. In embodiments, if a user is consistently making correctpredictions, the computing device 2702 may flag that user as a VIP user,which, in embodiments, may enable the computing device 2702 to weightresponses by the VIP user more heavily than responses from other users.In embodiments, if a user is consistently making incorrect predictions,the computing device 2702 may flag that user as an Incorrect User,which, in embodiments, may enable the computing device 2702 to weightresponses by the Incorrect User less heavily than responses from otherusers. In embodiments, if a VIP user response conflicts with anIncorrect User response, the computing device 2702 may discount theIncorrect User response and rely on the VIP response.

In embodiments, if the computing device 2702 has previously determinedthe reliability rating of a user, the computing device 2702 may updatethe reliability rating of the user. The updating of the reliabilityrating may be similar to the updating of the reliability ratingdescribed above in connection with the processes of FIGS. 22 and 23 ,the descriptions of which applying herein.

Referring to FIG. 26 , at step S2622, the computing device (e.g.computing device 2702) calculates at least one of: a price predictionand/or a volume prediction. The computation of at least one of priceprediction and/or a volume prediction may be based on one or more of thefollowing: the first market response (e.g. first market response 4002)the second market response (e.g. second market response 4102), themarket data (e.g. market data 3902), additional historical data, thirdparty information (e.g. additional information from external datasources), and/or identification information, to name a few. Inembodiments, the computing device may access, via one or more databasesand/or third parties to obtain one or more of: the first market response(e.g. first market response 4002) the second market response (e.g.second market response 4102), the market data (e.g. market data 3902),additional historical data, and/or third party information (e.g.additional information from external data sources). In embodiments, thecomputing device 2702 may access the first market response 4002, thesecond market response 4102, and/or the market data 3902 by receivingthe first market response 4002, the second market response 4102, and/orthe market data 3902 from one or more of: internal storage 1808A,external storage 1808B, memory/storage 1815, system memory 1804, and/orstorage 1903, to name a few. In embodiments, the additional historicaldata and/or third party information may be received by the computingdevice 2702 via network 1910 from one or more third party vendors and/orthird party sources. In embodiments, the additional historical dataand/or third party information may be already stored on one or moredatabases of the computing device 2702. In embodiments where theadditional information is already stored, the computing device 2702 mayregularly (e.g. once a day, week, month, quarter, year, etc.) receivethe additional information from third party vendors and/or third partysources and store that information on one or more databases of thecomputing device 2702. In embodiments, the computing device 2702 mayaccess and/or receive the stored identification information from one ormore of: internal storage 1808A, external storage 1808B, memory/storage1815, system memory 1804, and/or storage 1903, to name a few. based onat least the first market response and the second market response. Inembodiments, at step S2622, the computing device calculates at least oneof the price prediction and the volume prediction based on the firstmarket response and the second market response, as well as additionalinformation from external data sources.

In embodiments, the calculation of the price prediction and/or volumeprediction may be based on the first market response (e.g. first marketresponse 4002) and the second market response (e.g. second marketresponse 4102) provided by the first group of user devices (e.g. firstgroup of user devices 2708). In embodiments, the calculation of theprice prediction and/or volume prediction may be based on the firstmarket response (e.g. first market response 4002) and the second marketresponse (e.g. second market response 4102) provided by the second groupof user devices (e.g. the aforementioned group of user devicesassociated with reliable users).

In embodiments, the calculation may use the prediction information 4102Bthat was transmitted by the user devices associated with reliable users(the reliability rating, in embodiments, being based on the first marketresponse 4002 and/or the second market response 4102 that wastransmitted by user devices of the first group of user devices 2708).If, for example, the prediction information 4102B shows that users arepredicting that the First Market is going to increase in price and involume, the computing device 2702 may predict that the First Market willincrease in price and in volume. If, for example, the predictioninformation 4102B shows that the First Market is going to decrease inprice and in volume, the computing device 2702 may determine that theFirst Market will decrease in value and in volume. The specificcalculations regarding the prediction information 4102B may be thecalculations currently used in the respective industry of the marketassociated with the market prediction, using information received in thefirst market response (e.g. first market response 4002) and the secondmarket response (e.g. second market response 4102).

In embodiments, as mentioned above, the calculated market priceprediction and/or volume prediction may be based on one or more of thefollowing: the first market response (e.g. first market response 4002)the second market response (e.g. second market response 4102), themarket data (e.g. market data 3902), additional historical data, thirdparty information (e.g. additional information from external datasources), and/or identification information, to name a few. Additionaldata (e.g. additional historical data and/or third party information)may enable the computing device 2702 to make more accurate marketpredictions. The additional historical data may include historical dataspecific to the industry of the market and/or specific to the marketwhich was queried in the second market query 3906. For example,historical data may include: previous open stock prices, previous highstock prices, previous low stock prices, spot prices, futures prices,previously predicted higher prices, current stock prices, past stockprices, past volumes past performance of the market, past volume ofmarket, and/or past pricing of the market, to name a few. The thirdparty information, in embodiments, may include: additional historicaldata, earnings reports, price-to-earnings ratios, products associatedwith the market and whether the products are coming to market soon,tariffs, taxes, legal dispute information, and/or corporate informationregarding one or more corporations associated with the market and/or theindustry related to the market, to name a few.

In embodiments, the computing device 2702 may implement a machinelearning technique to calculate the market prediction. In embodiments,the machine learning technique may be based on one or more of thefollowing: the first market response (e.g. first market response 4002)the second market response (e.g. second market response 4102), themarket data (e.g. market data 3902), additional historical data, thirdparty information (e.g. additional information from external datasources), and/or identification information, to name a few. The machinelearning technique, in embodiments, may implement a machine learningalgorithm, such as supervised learning algorithms (e.g. classificationsupervised learning, regression supervised learning), unsupervisedlearning algorithms (e.g. association unsupervised learning, clusteringunsupervised learning, dimensionality reduction unsupervised learning),reinforcement learning algorithms (e.g. through trial and error),semi-supervised algorithms, Naïve Bayes Classifier Algorithm, K MeansClustering Algorithm, Support Vector Machine Algorithm, AprioriAlgorithm, Linear Regression, Logistic Regression, Artificial NeuralNetworks, Random Forests, Decision Trees, and/or Nearest Neighbors, toname a few. In embodiments, the machine learning technique may be a deeplearning technique, which may be based on learning data representationsas opposed to task-specific algorithms. The deep learning technique maybe supervised, semi-supervised, and/or unsupervised. In embodiments, thestock market prediction calculation may be performed by using a neuralnetwork technique, which may include a deep learning neural networktechnique (e.g. DNN). A DNN may be an artificial neural network withmultiple layers between the input (e.g. the first market response 4002and the second market response 41020) and output (e.g. the priceprediction and/or volume prediction).

In embodiments, once the price prediction and/or volume prediction iscalculated, the computing device 2702 may generate a price predictionmessage (e.g. price prediction 4202) and/or a volume prediction message(e.g. volume prediction 4204). Referring to FIG. 42 , the priceprediction 4202, in embodiments, may state “We believe the price ofStock A will increase to $200.00 per share within the next quarter.” Inembodiments, the price prediction 4202 may include one or more of thefollowing: the price of an asset or assets within the market, the amountof users that sent the first market response 4002, the amount of usersthat sent the second market response 4102, the amount of reliable users,the amount of reliable users that gave information that was used in theprice prediction 4202, and/or a recommendation based on the priceprediction 4202. In embodiments, the volume prediction 4204, may state“We believe 1,000 shares of Stock A will be purchased within the nextquarter.” In embodiments, the volume prediction 4204 may include one ormore of the following: the volume of an asset or assets within themarket, the amount of users that sent the first market response 4002,the amount of users that sent the second market response 4102, theamount of reliable users, the amount of reliable users that gaveinformation that was used in the volume prediction 4204, and/or arecommendation based on the volume prediction 4204.

Referring back to FIG. 26 , at step S2624, the computing device (e.g.computing device 2702) transmits at least one of the price prediction(e.g. price prediction 4202) and the volume prediction (e.g. volumeprediction 4204) to the plurality of users (e.g. plurality of users3006). Referring to FIG. 42 , in embodiments, the generated priceprediction 4202 and/or volume prediction 4204 may be transmitted, vianetwork 1910, to one or more of: the plurality of user 3006, the firstgroup of user devices 2708, the second group of user devices, and/or agroup of users devices associated with users who provided informationthat was used in either the price prediction 4202 and/or volumeprediction 4204. In embodiments, the price prediction 4202 and/or volumeprediction 4204 may not be sent to unreliable users. In thoseembodiments, a notification message may be generated and transmitted bythe computing device 2702 to user devices associated with the unreliableusers. The notification may state why the stock market prediction wasnot sent to the unreliable user (e.g. because your information was notreliable).

In embodiments, users may be incentivized to respond to messages sentfrom the computing device 2702 (e.g. event stimulus message 2704, newsstimulus message 2706, first market query 3102, second market query3104, first stock market query 3402, second stock market query 3404,third stock market query 3702, first market response 3904, and/or secondmarket response 3906, to name a few). For example, a user may receive apoint for every responsive message sent to the computing device 2702(responsive messages may include first response 2802, first response2902, first market response 3202, second market response 3302, firststock market response 3502, second stock market response 3602, thirdstock market response 3802, first market response 4002, and/or secondmarket response 4102, to name a few). These points, in embodiments, maybe compared to other user point totals. For example, point totals may becompared within groups (e.g. the users associated with the first groupof user devices 2708) or compared globally (e.g. the users associatedwith the plurality of devices 3006). The top point earners may receive atitle for being the top point earner of a certain period of time (e.g.all time, a year, a month, a week, a day, etc., to name a few). If auser has a title, or has earned a title, the computing device 2702 maygenerate and send a message to a user device associated with therelevant user stating, for example, “Congrats Mr. Responsive, you haveresponded to the most stimulus messages this week.” As anotherincentive, if a user loses a title, the computing device 2702 maygenerate and send a message to a user device associated with therelevant user, stating, for example, “John just took your Mr. Responsivetitle, why not respond to a new stimulus message and reclaim yourtitle?”

As another example, users may receive points based on their respectivereliability rating. As with the above example, the points, may becompared to other user point totals. For example, point totals may becompared within groups (e.g. the users associated with the first groupof user devices 2708) or compared globally (e.g. the users associatedwith the plurality of devices 3006). The top point earners may receive atitle for being the top point earner of a certain period of time (e.g.all time, a year, a month, a week, a day, etc., to name a few). If auser has a title, or has earned a title, the computing device 2702 maygenerate and send a message to a user device associated with therelevant user stating, for example, “Congrats Mr. Reliable, you haveresponded to the most stimulus messages this week.” As anotherincentive, if a user loses a title, the computing device 2702 maygenerate and send a message to a user device associated with therelevant user, stating, for example, “Jason just took your Mr. Reliabletitle, get your reliability rating up and take it back!”

In embodiments, other incentives which may be offered, may includefinancial incentives, which may include, money, coupons, discounts, freememberships, and/or free items in a giveaway (e.g. a t-shirt, hat,cufflinks etc., to name a few). Other exemplary factors that may be usedto give incentives may include, for example, how fast a user responds,how often a user responds, how complete each user response is, how manyimages are sent with responses, how many audio files are sent withresponses, how many videos are sent with responses, how often a user iswithin a predetermined radius for responses, and/or how many referencesthe user receives in prepared news stories, timelines, and/orpredictions, to name a few.

The steps of the process described in connection with FIG. 26 , inembodiments, may be rearranged or omitted.

In embodiments, the present invention generally relates to a unique andnon-routine method for identifying an unreliable user(s) of a networkand/or using a user response(s) in the network to provide an accuratetimeline of an event. As further described below, at least oneembodiment of the present invention includes unconventional andnon-routine method steps that specify how to collect responses from aplurality of users, analyze those responses to determine a reliabilityrating based on the response information and create an accurate timelineof an event based on the responses while accounting for unreliableresponses.

In embodiments, a computing device connected to the network may beconfigured to assign reliability ratings (e.g., reliable, unreliable orgraded reliability ratings) to users based on the accuracy of theinformation provided in their responses, wherein a user providingaccurate information in a response is reliable and a user providinginaccurate information in a response is unreliable. Based on thesereliability ratings, in embodiments, a decision is made as to whether arespective user is unreliable. In embodiments, responses provided byreliable users are weighted more heavily in creating a timeline thanresponse from unreliable users, such that the net result is a morereliable and accurate timeline. These unique and non-routine steps offera technical solution to the technical problem posed by deceitful usersin network applications that rely on user responses.

Various embodiments of the invention provide generally for a real-timeevent transcription system and related methods. Some embodiments providefurther for utilizing cheating detection and compensation methods whileproviding synchronized user engagement. Additional embodiments providefurther ability to send, target or tailor advertisements, marketing, orother promotional material to a plurality of users based on an analysisof the users (individually or in any grouping) affinity to particularteams, actors, celebrities, or other entities and a further analysisthat predicts the reaction of those users to events as they transpire.

For example, the present invention relates to a method of generating anaccurate news report based on information provided by one or more usersof a plurality of users of an electronic computer network, the methodincluding steps of: (a) receiving, by a computing device, identificationinformation associated with each user of the plurality of users of theelectronic computer network; (b) storing, by the computing device in oneor more databases, the identification information; (c) generating, bythe computing device, a first stimulus message related to an event; (d)transmitting, by the computing device, the first stimulus message to afirst group of user devices associated with a first group of users ofthe plurality of users; (e) receiving, by the computing device from oneor more user devices of the first group of user devices, a firstresponse, wherein the first response includes: (i) user informationspecific to the respective user associated with the respective userdevice that transmits the first response; (ii) responsive informationrelated to the event; (iii) location information associated with alocation of the respective user device associated with the respectiveuser; and (iv) a timestamp; (f) storing, by the computing device, thefirst response in the one or more databases; (g) determining, by thecomputing device, authenticity of the first response based on one ormore of the responsive information, the location information and thetimestamp; (h) assigning, by the computing device, a reliability ratingto the respective user based the first response by performing steps of:(i) assigning, by the computing device, the reliability rating to be areliable rating when the location information is consistent with alocation of the event and the timestamp indicates acceptable delay; and(ii) assigning, by the computing device, the reliability rating to be anunreliable rating when the location information is inconsistent with thelocation of the event or the timestamp indicates an unacceptable delay;(i) storing, by the computing device in the one or more databases, thereliability rating; (j) identifying, by the computing device, one ormore reliable users based on the reliability rating; (k) selecting, bythe computing device, the responsive information associated with the oneor more reliable users; and (l) generating, by the computer system, anews report based on the selected responsive information.

In embodiments, the first stimulus message includes machine readableinstructions to present an inquiry message relating to a past eventelement of the event on the one or more user devices of the first groupof user devices.

In embodiments, the first stimulus message includes machine readableinstructions to present an inquiry message relating to a future eventelement of the event on the one or more user devices of the first groupof user devices.

In embodiments, the user information in the first response from eachuser device of the one or more user devices of the first group of userdevices includes proximity information regarding each user device, inembodiments, the method further includes the step (g) of determining theauthenticity of the first response is based on the proximity of therespective user device to other user devices.

In embodiments, the method further includes a step of determining, bythe computing device, a time lag between occurrence of a particularevent element of the event and receipt of the respective first responsefrom each respective user device of the one or more user devices fromthe first group of user devices including information relating to theparticular event element. In embodiments, the user informationassociated with the respective user includes information regarding aconnection of the user device associated with the respective user to theelectronic computer network and in embodiments, the method furtherincludes the step (g) of determining the authenticity of the firstresponse further includes a step of comparing, by the computing device,the determined time lag to an expected time lag for the connection ofthe user device associated with the respective user to the electroniccomputer network. In embodiments, the step (h) of assigning thereliability rating to the respective user further includes a step ofassigning, by the computing device, the reliability rating to therespective user to be an unreliable rating when the determined time lagof the first response from the user device associated with therespective user is outside upper and lower thresholds of the expectedtime lag.

In embodiments, the first stimulus message includes machine readableinstructions to present an inquiry message on each user device of thefirst group of user devices regarding a past event element of the eventthat was modified or did not take place. In embodiments, the step (h) ofassigning the reliability rating further includes a step of assigning,by the computing device, the reliability rating to the respective userto be an unreliable rating when the first response from the user deviceassociated with the respective user includes first response informationconfirming occurrence of the past event element that was modified or didnot take place.

In embodiments, the method further includes a step of creating, by thecomputing device, a timeline of the event based on the first responsefrom the one or more user devices of the first group of user devices. Inembodiments, the step of creating the timeline further includes a stepof weighing, by the computing device, first response information fromthe first response received from the one or more user devices of thefirst group of user devices based on the reliability rating of therespective user associated with the user device providing the firstresponse, the first response information from the first response fromthe respective user devices associated with the users with the reliablerating is weighted more heavily than the first response information fromthe first response from the respective user devices associated with theusers with the unreliable rating.

In embodiments, the method further includes steps of: (m) transmitting,from the computing device, after a first predetermined period of time, asecond stimulus message related to the event to the first group of userdevices, the second stimulus message including information related tothe event and machine readable instructions to present a second messageto each user of the first group of users on each user device of thefirst group of user devices that prompts a second response from eachuser; (n) receiving, by the computing device via the electronic computernetwork, the second response from one or more user devices of the firstgroup of user devices, the second response including second responseinformation related to the second message and the user informationspecific to the user associated with the respective user device; (o)storing, by the computing device, the second response in memoryaccessible by the computing device; (p) determining, by the computingdevice, authenticity of the second response from each of the one or moreuser devices of the first group of user devices, by performing at leastthe following steps: (i) determining, by the computing device, whetherthe second response includes the second response information thatcorresponds to activity in the event; and (ii) determining, by thecomputing device, whether the second response includes the secondresponse information relating to event elements available to therespective user when the second message was displayed on the user deviceassociated with the respective user; and (q) updating, by the computingdevice, the reliability rating for each user of the one or more userdevices based on the authenticity of the second response received fromthe user device associated with the respective user. In embodiments, thestep (q) of updating the reliability rating includes steps of: (i)assigning, by the computing device, a reliable rating to the respectiveuser when the second response from the user device associated with therespective user includes the second response information thatcorresponds to activity in the event; (ii) assigning, by the computingdevice, a reliable rating to the respective user when the secondresponse received from the user device associated with the respectiveuser includes the second response information available to therespective user of the user device associated with the respective userwhen the second message was displayed; (iii) assigning, by the computingdevice, an unreliable rating to the respective user when the secondresponse received from the user device associated with the respectiveuser includes the second response information that does not correspondto activity in the event; and (iv) assigning, by the computing device,an unreliable rating to the respective user when the second responsereceived from the user device associated with the respective userincludes the second response information that was not available to therespective user when the second message was displayed. In embodiments,the step (q) of updating the reliability rating further includes stepsof: (i) assigning, by the computing device, a reliable rating when therespective user received the reliable rating for the first predeterminedperiod of time followed by a second reliable rating, or the reliablerating for the first predetermined period of time followed by a secondunreliable rating; and (ii) assigning, by the computing device, anunreliable rating when the respective user received the unreliablerating for the first predetermined period of time followed by a secondunreliable rating, or the unreliable rating for the first predeterminedperiod of time followed by a second reliable rating. In embodiments, themethod may further include the following steps: creating, by thecomputing device, a timeline of the event based on the first responsefrom the one or more user devices of the first group of user devices;and updating, by the computing device, the timeline of the event basedon the second response after the first predetermined period of time. Inembodiments, the step of updating the timeline further includes a stepof weighting, by the computing device, the second response informationin the second response received from the one or more user devices basedon the reliability rating of the respective user associated with therespective user device providing the second response, the secondresponse information in the second response from the user devicesassociated with the respective users with the reliable rating isweighted more heavily than the second response information in the secondresponse from the user devices associated with the respective users withthe unreliable rating. In embodiments, the step of updating the timelinefurther includes a step of weighting, by the computing device, thesecond response information in the second response received from the oneor more user devices based on the reliability rating of the respectiveuser associated with the respective user device providing the secondresponse when the second response was initiated from a user device at alocation geographically proximal to the event, the second responseinformation in the second response from the user devices associated withthe respective users with the reliable rating is weighted more heavilythan the second response information in the second response from theuser devices associated with the respective users with the unreliablerating. In embodiments, the step of updating the timeline furtherincludes a step of weighting, by the computing device, the secondresponse information in the second response received from the one ormore user devices based on the reliability rating of the respective userassociated with the respective user device providing the second responsewhen the second response was initiated from a user device locatedgeographically proximal to other user devices with similar reliabilityratings, and the second response information in the second response fromthe user devices associated with the respective users with the reliableratings is weighted more heavily than the second response information inthe second response from the user devices associated with the respectiveusers with the unreliable rating.\

In embodiments, the timestamp includes: (A) a first timestamp indicatingwhen the first response was transmitted; and (B) a second timestampindicating when the first response was received by the computing device.

The present invention also relates to a method of determining accuracyof a news report based on information provided by one or more users of aplurality of users of an electronic computer network, the methodincluding steps of: (a) receiving, by a computing device, identificationinformation associated with each user of the plurality of users of theelectronic computer network; (b) storing, by the computing device in oneor more databases, the identification information; (c) generating, bythe computing device, a first stimulus message related to the newsreport; (d) transmitting, by the computing device, the first stimulusmessage to a first group of user devices associated with a first groupof users of the plurality of users; (e) receiving, by the computingdevice from one or more user devices of the first group of user devices,a first response, wherein the first response includes: (i) userinformation specific to the respective user associated with therespective user device that transmits the first response; (ii)responsive information related to the news report; (iii) locationinformation associated with a location of the respective user device;and (iv) a timestamp; (f) storing, by the computing device, the firstresponse in the one or more databases; (g) determining, by the computingdevice, authenticity of the first response based on one or more of theresponsive information, the location information and the timestamp; (h)assigning, by the computing device, a reliability rating to therespective user based the first response by performing steps of: (i)assigning, by the computing device, the reliability rating to be areliable rating when the location information is consistent with alocation associated with the news report and the timestamp indicatesacceptable delay; (ii) assigning, by the computing device, thereliability rating to be an unreliable rating when the locationinformation is inconsistent with the location associated with the newsreport or the timestamp indicates an unacceptable delay; (i) storing, bythe computing device in the one or more databases the reliabilityrating; (j) identifying, by the computing device, one or more reliableusers based on the reliability rating; (k) selecting, by the computingdevice, the responsive information associated with the one or morereliable users; (l) determining, by the computing device, a news reportreliability rating based on the selected responsive informationassociated with the one or more reliable users; (m) transmitting, by thecomputing device, the news report reliability rating to the plurality ofusers.

In embodiments, the user information in the first response from eachuser device of the one or more user devices of the first group of userdevices includes proximity information regarding each user device andthe step (g) of determining the authenticity of the first response isbased on the proximity of the respective user device to other userdevices.

In embodiments, the method further includes a step of determining, bythe computing device, a time lag between occurrence of a particularevent element of the news report and receipt of the respective firstresponse from each respective user device of the one or more userdevices from the first group of user devices including informationrelating to the particular event element. In embodiments, the userinformation associated with the respective user includes informationregarding a connection of the user device associated with the respectiveuser to the electronic computer network and the step (g) of determiningthe authenticity of the first response further includes a step ofcomparing, by the computing device, the determined time lag to anexpected time lag for the connection of the user device associated withthe respective user to the electronic computer network. In embodiments,the step (h) of assigning the reliability rating to the respective userfurther includes a step of assigning, by the computing device, thereliability rating to the respective user to be an unreliable ratingwhen the determined time lag of the first response from the user deviceassociated with the respective user is outside upper and lowerthresholds of the expected time lag.

In embodiments, the first stimulus message includes machine readableinstructions to present an inquiry message on each user device of thefirst group of user devices regarding a past event element of the newsreport that was modified or did not take place. In embodiments, the step(h) of assigning the reliability rating to the respective user furtherincludes a step of assigning, by the computing device, the reliabilityrating to the respective user to be an unreliable rating when the firstresponse from the user device associated with the respective userincludes first response information confirming occurrence of the pastevent element that was modified or did not take place.

In embodiments, the method further includes a step of creating, by thecomputing device, a timeline of the event based on the first responsefrom the one or more user devices of the first group of user devices.

In embodiments, the step of creating the timeline further includes astep of weighting, by the computing device, first response informationfrom the first response received from the one or more user devices ofthe first group of user devices based on the reliability rating of therespective user associated with the user device providing the firstresponse, wherein the first response information from the first responsefrom the respective user devices associated with the users with thereliable rating is weighted more heavily than the first responseinformation from the first response from the respective user devicesassociated with the users with the unreliable rating.

In embodiments, the method further includes the steps of: (n)transmitting, from the computing device, after a first predeterminedperiod of time, a second stimulus message related to the news report tothe first group of user devices, the second stimulus message includinginformation related to the news report and machine readable instructionsto present a second message to each user of the first group of users oneach user device of the first group of user devices that prompts asecond response from each user; (o) receiving, by the computing devicevia the electronic computer network, the second response from one ormore user devices of the first group of user devices, the secondresponse including second response information related to the secondmessage and the user information specific to the user associated withthe respective user device; (p) storing, by the computing device, thesecond response in memory accessible by the computing device; (q)determining, by the computing device, authenticity of the secondresponse from each of the one or more user devices of the first group ofuser devices, by performing at least the following steps: (i)determining, by the computing device, whether the second responseincludes the second response information that corresponds to an activityevent element in the news report; and (ii) determining, by the computingdevice, whether the second response includes the second responseinformation relating to news report elements available to the respectiveuser when the second message was displayed on the user device associatedwith the respective user; and (r) updating, by the computing device, thereliability rating for each user of the one or more user devices basedon the authenticity of the second response received from the user deviceassociated with the respective user. In embodiments, the step (r) ofupdating the reliability rating includes steps of: (i) assigning, by thecomputing device, a reliable rating to the respective user when thesecond response from the user device associated with the respective userincludes the second response information that corresponds to theactivity event element in the news report; (ii) assigning, by thecomputing device, a reliable rating to the respective user when thesecond response received from the user device associated with therespective user includes the second response information available tothe respective user of the user device associated with the respectiveuser when the second message was displayed; (iii) assigning, by thecomputing device, an unreliable rating to the respective user when thesecond response received from the user device associated with therespective user includes the second response information that does notcorrespond to the activity event element in the news report; and (iv)assigning, by the computing device, an unreliable rating to therespective user when the second response received from the user deviceassociated with the respective user includes the second responseinformation that was not available to the respective user when thesecond message was displayed. In embodiments, the step (r) of updatingthe reliability rating further includes steps of: (i) assigning, by thecomputing device, a reliable rating when the respective user receivedthe reliable rating for the first predetermined period of time followedby a second reliable rating, or the reliable rating for the firstpredetermined period of time followed by a second unreliable rating; and(ii) assigning, by the computing device, an unreliable rating when therespective user received the unreliable rating for the firstpredetermined period of time followed by a second unreliable rating, orthe unreliable rating for the first predetermined period of timefollowed by a second reliable rating. In embodiments, the method furtherincludes the steps of: creating, by the computing device, a timeline ofthe news report based on the first response from the one or more userdevices of the first group of user devices; and updating, by thecomputing device, the timeline of the news report based on the secondresponse after the first predetermined period of time. In embodiments,the step of updating the timeline further includes a step of weighting,by the computing device, the second response information in the secondresponse received from the one or more user devices based on thereliability rating of the respective user associated with the respectiveuser device providing the second response, the second responseinformation in the second response from the user devices associated withthe respective users with the reliable rating is weighted more heavilythan the second response information in the second response from theuser devices associated with the respective users with the unreliablerating. In embodiments, the step of updating the timeline furtherincludes a step of weighting, by the computing device, the secondresponse information in the second response received from the one ormore user devices based on the reliability rating of the respective userassociated with the respective user device providing the second responsewhen the second response was initiated from a user device at a locationgeographically proximal to an event element of the news report, and thesecond response information in the second response from the user devicesassociated with the respective users with the reliable rating isweighted more heavily than the second response information in the secondresponse from the user devices associated with the respective users withthe unreliable rating. In embodiments, the step of updating the timelinefurther includes a step of weighting, by the computing device, thesecond response information in the second response received from the oneor more user devices based on the reliability rating of the respectiveuser associated with the respective user device providing the secondresponse when the second response was initiated from a user devicelocated geographically proximal to other user devices with similarreliability ratings, and the second response information in the secondresponse from the user devices associated with the respective users withthe reliable ratings is weighted more heavily than the second responseinformation in the second response from the user devices associated withthe respective users with the unreliable rating.

In embodiments, the timestamp includes: (A) a first timestamp indicatingwhen the first response was transmitted; and (B) a second timestampindicating when the first response was received by the computing device.

The present invention also relates to a method of predicting financialmarket conditions based on information provided by one or more users ofa plurality of users of an electronic computer network, the methodincluding steps of: (a) receiving, by a computing device, identificationinformation associated with each user of a plurality of users of theelectronic computer network; (b) storing, by the computing device in oneor more databases, the identification information; (c) generating, bythe computing device, a first market query related to past financialmarket conditions; (d) transmitting, by the computing device to at leasta first group of user devices associated with a first group of users ofthe plurality of users of the electronic computer network, the firstmarket query; (e) receiving, by the computing device from at least aplurality of user devices of the first group of user devices, a firstmarket response, the first market response including: (i) userinformation unique to the respective user associated with the respectiveuser device providing the first market response; (ii) past marketinformation related to prior market conditions; and (iii) a timestamp;(f) storing, by the computing device in the one or more databases, thefirst market response of each user device of the plurality of userdevices of the first group of user devices from which the first marketresponse was received; (g) generating, by the computing device, a secondmarket query related to future market conditions; (h) transmitting, bythe computing device, the second market query to the first group of userdevices; (i) receiving, by the computing device from at least aplurality of user devices of the first group of user devices, a secondmarket response, the second market response including: (i) userinformation unique to the respective user associated with the respectiveuser device providing the second market response; (ii) a prediction forthe future market conditions; and (iii) a second timestamp; (j) storing,by the computing device in the one or more databases, the second marketresponse of each user device of the plurality of user devices from whichthe second market response was received; (k) accessing, by the computingdevice, at least the first market response and the second marketresponse provided by each user device of the first group of userdevices; (l) calculating, by the computing device, a market predictionrelated to the future market conditions based on at least the firstmarket response and the second market rely provided by the first groupof user devices.

In embodiments, the past financial market conditions include past priceinformation or past volume information.

In embodiments, the calculating step (l) includes calculating, by thecomputing device, the market prediction related to the future marketconditions based on the first market response and the second marketresponse provided by the first group of user devices, additionalhistorical data and additional information from external data sources.

In embodiments, the step (l) of calculating the market prediction isperformed by using a machine learning technique based on the at leastthe first market response and the second market response provided by thefirst group of user devices as an input.

In embodiments, the step (l) of calculating the market prediction isperformed by using a neural network technique based on the at least thefirst market response and the second market response provided by thefirst group of user devices as an input.

In embodiments, the neural network technique includes a deep learningneural network technique.

In embodiments, the first market query includes machine readableinstructions to present an inquiry message relating to the pastfinancial market conditions on the one or more user devices of the firstgroup of user devices.

In embodiments, the second market query includes machine readableinstructions to present an inquiry message relating to the future marketconditions on the one or more user devices of the first group of userdevices.

In embodiments, the method further includes a step of determining, bythe computing device, a time lag between occurrence of a particularmarket event element of the past financial market conditions and receiptof the respective first market response from each respective user deviceof the one or more user devices from the first group of user devicesincluding information relating to the particular event element. Inembodiments, the method further includes, determining authenticity ofthe first market response by performing a step of comparing, by thecomputing device, the determined time lag to an expected time lag for aconnection of the user device associated with the respective user to theelectronic computer network. In embodiments, the method furtherincludes, assigning, by the computing device, an unreliable rating tothe respective user when the determined time lag of the first responsefrom the user device associated with the respective user is outsideupper and lower thresholds of the expected time lag.

In embodiments, the first market query includes machine readableinstructions to present an inquiry message on each user device of thefirst group of user devices regarding a past market event element of thepast financial market conditions that was modified or did not takeplace. In embodiments, the method further includes, assigning, by thecomputing device, an unreliable rating to the respective user when thefirst market response from the user device associated with therespective user includes first market response information confirmingoccurrence of the past market event element that was modified or did nottake place.

In embodiments, the first timestamp includes: (A) a third timestampindicating when the first market response was transmitted; and (B) afourth timestamp indicating when the first market response was receivedby the computing device.

In embodiments, the second timestamp includes: (A) a fifth timestampindicating when the second market response was transmitted; and (B) asixth timestamp indicating when the second market response was receivedby the computing device.

The present invention also relates a method of predicting stock marketconditions based on information provided by one or more users of aplurality of users of an electronic computer network, the methodincluding steps of: (a) receiving, by a computing device, identificationinformation associated with each user of a plurality of users of theelectronic computer network; (b) storing, by the computing device in oneor more databases, the identification information; (c) generating, bythe computing device, a first stock market query related to prior stockmarket conditions; (d) transmitting, by the computing device to a firstgroup of user devices associated with a first group of users of theplurality of users of the electronic computer network, the first stockmarket query; (e) receiving, by the computing device from a plurality ofuser devices of the first group of user devices, a first stock marketresponse, the first stock market response including: (i) userinformation unique to the respective user associated with the respectiveuser device providing the first stock market response; (ii) past stockmarket information related to the prior stock market conditions; and(iii) a timestamp; (f) storing, by the computing device in the one ormore databases, the first stock market response of each user device ofthe plurality of user devices of the first group of user devices fromwhich the first stock market response was received; (g) generating, bythe computing device, a second stock market query related to futurestock market conditions; (h) transmitting, by the computing device, thesecond stock market query to the first group of user devices; (i)receiving, by the computing device, a second stock market response fromat least a plurality of user devices of the first group of user devices,the second stock market response including: (i) user information uniqueto the respective user associated with the respective user deviceproviding the second stock market response; (ii) a prediction for thefuture stock market conditions; and (iii) a second timestamp; (h)storing, by the computing device in the one or more databases, thesecond stock market response; (i) accessing, by the computing device, atleast the first stock market response and the second stock marketresponse provided by each user device of the first group of userdevices; (j) calculating, by the computing device, a stock marketprediction related to the future stock market conditions based on atleast the first stock market response and second stock market responseprovided by the first group of user devices.

In embodiments, the first stock market query includes a query relatingto past price information for a particular stock, past volumeinformation for a particular stock, or past price/volume information fora sector.

In embodiments, the step (j) of calculating the stock market predictionis performed by using a machine learning technique based on the at leastthe first stock market response and second stock market responseprovided by the first group of user devices as an input.

In embodiments, the step (j) of calculating the stock market predictionis performed by using a neural network technique based on the at leastthe first stock market response and second stock market responseprovided by the first group of user devices as an input.

In embodiments, the neural network technique includes a deep learningneural network technique.

In embodiments, the method of predicting stock market conditions furtherincludes (k) detecting, by the computing device, a trading pattern byperforming steps of: (i) generating, by the computing device, a thirdstock market query related to past transactions; (ii) transmitting, bythe computing device, the third stock market query to the first group ofuser devices; (iii) receiving, by the computing device, a third stockmarket response from at least a plurality of user devices of the firstgroup of user devices, the third stock market response including: (A)user information unique to the respective user associated with therespective user device providing the third stock market response; (B)stock ID information for a particular stock; (C) stock price informationfor the particular stock; (D) buy/sell data information for theparticular stock; and (E) quantity information for the particular stock;(iv) storing, by the computing device in the one or more databases, thethird stock market response; (v) accessing, by the computing device, atleast the first stock market response, the second stock market responseand the third stock market response provided by each user device of thefirst group of user devices; (vi) determining, by the computing device,the trading pattern based on at least the first stock market response,the second stock market response and the third stock market responseprovided by the first group of user devices. In embodiments, the step(k)(vi) of determining the trading pattern is performed by using amachine learning technique based on the at least the first stock marketresponse, the second stock market response and the third stock marketresponse provided by the first group of user devices as an input. Inembodiments, the step (k)(vi) of determining the trading pattern isperformed by using a neural network technique based on the at least thefirst stock market response, the second stock market response and thethird stock market response provided by the first group of user devicesas an input. In embodiments, the neural network technique includes adeep learning neural network technique.

In embodiments, the method of predicting stock market conditions furtherincludes steps of (l) comparing, by the computing device, the determinedtrading pattern to other users; and (m) generating, by the computingdevice, trading suggestions for the other users based on the result ofthe comparing step.

In embodiments, the first stock market query includes machine readableinstructions to present an inquiry message relating to the prior stockmarket conditions on the one or more user devices of the first group ofuser devices.

In embodiments, the second stock market query includes machine readableinstructions to present an inquiry message relating to the future stockmarket conditions on the one or more user devices of the first group ofuser devices.

In embodiments, the method further includes the step of determining, bythe computing device, a time lag between occurrence of a particularstock market event element of the prior stock market conditions andreceipt of the respective first stock market response from eachrespective user device of the one or more user devices from the firstgroup of user devices including information relating to the particularstock market event element. In embodiments, the method further includesthe step of determining authenticity of the first stock market responseby performing a step of comparing, by the computing device, thedetermined time lag to an expected time lag for a connection of the userdevice associated with the respective user to the electronic computernetwork. In embodiments, the method further includes the step ofassigning, by the computing device, an unreliable rating to therespective user when the determined time lag of the first stock marketresponse from the user device associated with the respective user isoutside upper and lower thresholds of the expected time lag.

In embodiments, the first stock market query includes machine readableinstructions to present an inquiry message on each user device of thefirst group of user devices regarding a past stock market event elementof the prior stock market conditions that was modified or did not takeplace. In embodiments, the method further includes the step ofassigning, by the computing device, an unreliable rating to therespective user when the first stock market response from the userdevice associated with the respective user includes first stock marketresponse information confirming occurrence of the past stock marketevent element that was modified or did not take place.

In embodiments, the first timestamp includes: (A) a third timestampindicating when the first stock market response was transmitted; and (B)a fourth timestamp indicating when the first stock market response wasreceived by the computing device.

In embodiments, second timestamp includes: (A) a fifth timestampindicating when the second stock market response was transmitted; and(B) a sixth timestamp indicating when the second stock market responsewas received by the computing device.

The present invention also relates to a method of gathering opinioninformation provided by one or more users of a plurality of users of anelectronic computer network, the method including steps of: (a)receiving, by a computing device, identification information associatedwith each user of a plurality of users of the electronic computernetwork; (b) storing, by the computing device in one or more databases,the identification information; (c) transmitting, by the computingdevice to at least a first group of user devices associated with a firstgroup of users of the plurality of users of the electronic computernetwork, market data, the market data including: (i) past priceinformation; (ii) past volume information; and (iii) a first timestamp;(d) generating, by the computing device, a first market query related tothe market data; (e) transmitting, by the computing device, the firstmarket query to one or more user devices of the first group of userdevices; (f) receiving, by the computing device, a first market responsefrom at least one user device of the first group of user devices, thefirst market response including: (i) user identification informationunique to the respective user device and associated with the respectiveuser associated with the respective user device; (ii) impressionsentiment information related to the respective user's impression of themarket data; and (iii) a second timestamp; (g) generating, by thecomputing device, a second market query related to future marketparameters; (h) transmitting, by the computing device, the second marketquery to one or more user devices of the first group of user devices;(i) receiving, by the computing device, a second market response from atleast one respective user device of the first group of user devices, thesecond market response including: (i) user identification informationunique to the respective user associated with the respective userdevice; (ii) prediction information related to at least one of futureprice information and future volume information; and (iii) a thirdtimestamp; (j) storing, by the computing device in the one or moredatabases, the second market response; (k) calculating, by the computingdevice, at least one of a price prediction and a volume prediction basedon at least the first market response and the second market response;and (l) transmitting, by the computing device, at least one of the priceprediction and the volume prediction to the plurality of users.

In embodiments, the calculating step (k) includes calculating, by thecomputing device, at least one of the price prediction and the volumeprediction based on the first market response, the second marketresponse, and additional information from external data sources. Inembodiments, the method further includes a step of determiningauthenticity of the first market response by performing a step ofcomparing, by the computing device, the determined time lag to anexpected time lag for a connection of the user device associated withthe respective user to the electronic computer network. In embodiments,the method further includes a step of assigning, by the computingdevice, an unreliable rating to the respective user when the determinedtime lag of the first market response from the user device associatedwith the respective user is outside upper and lower thresholds of theexpected time lag.

In embodiments, the step (k) of calculating the at least one of theprice prediction and the volume prediction is performed by using amachine learning technique based on the at least the first marketresponse and the second market response as an input.

In embodiments, the step (k) of calculating the at least one of theprice prediction and the volume prediction is performed by using aneural network technique based on the at least the first market responseand the second market response as an input. In embodiments, the neuralnetwork technique includes a deep learning neural network technique.

In embodiments, the first market query includes machine readableinstructions to present an inquiry message relating to the market dataon the one or more user devices of the first group of user devices.

In embodiments, the second market query includes machine readableinstructions to present an inquiry message relating to the future marketparameters on the one or more user devices of the first group of userdevices.

In embodiments, the method further includes a step of determining, bythe computing device, a time lag between occurrence of a particularmarket event element of the market data and receipt of the respectivefirst market response from each respective user device of the one ormore user devices from the first group of user devices includinginformation relating to the particular market event element.

In embodiments, the first market query includes machine readableinstructions to present an inquiry message on each user device of thefirst group of user devices regarding a past market event element of themarket data that was modified or did not take place. In embodiments, themethod further includes a step of assigning, by the computing device, anunreliable rating to the respective user when the first market responsefrom the user device associated with the respective user includes firstmarket response information confirming occurrence of the past marketevent element that was modified or did not take place.

In embodiments, the second timestamp includes: (A) a fourth timestampindicating when the first market response was transmitted; and (B) afifth timestamp indicating when the first market response was receivedby the computing device.

In embodiments, the third timestamp includes: (A) a sixth timestampindicating when the second market response was transmitted; and (B) aseventh timestamp indicating when the second market response wasreceived by the computing device.

Some embodiments provide for a system comprising at least one computerprocessor and associated machine instructions, the system configured toallow a plurality of entities (humans and/or electronic and/ormechanical), who (or which) may be referred to as “users” herein, who(or which) are observing or interacting with some event or task (live orotherwise) to interact with a stream of stimuli generated over thecourse of the event, said interactions happening with, by and betweenmultiple entities including other users, organizations, eventaffiliates, and to allow the entities or users to interact with thestreamed stimuli in a fair and accurate manner whereby each users'interactions are precisely recorded and accurately timestamped,accounting for various delays that may cause different users toencounter the stimuli at different times.

One or more embodiments of the invention provide for various methods toprevent users from cheating (for example, by users using the delays totheir advantage) and for various methods for detecting and synchronizingthe stimuli. Embodiments provide for additional methods to determine andsynchronize stimuli and reaction timing across a plurality ofgeographical locations, transmission methods and media.

Embodiments of the present invention provide further for a system andmethods that can accurately and quickly predict the past, providevaluable insights into future events, and determine the likelihood ofcheating by an individual user in an online game or other application.By crowdsourcing user inputs and integrating additional signals from aplurality of additional sources, data can be gathered for eachoccurrence throughout the course of an event, and the data furtherprocessed to enable reconstruction of past occurrences and gathering ofinsights regarding these occurrences. These insights, in turn, can beused for various applications in multiple embodiments, some of which aredescribed herein.

Various embodiments of the invention provide for generating a transcriptof any event that is concurrently observed by a plurality of entities ina manner whereby each such entity can react to a plurality of stimuliand thereby interact with the system contemplated herein such that anaccurate transcript will be produced by the system. Various embodimentscan provide further benefit by being able to detect any of attemptedfraud or cheating by any such entity. Additionally, embodiments canprovide benefit by being able to accomplish the foregoing steps withoutthe requirement for a physical presence at the location of the eventand/or without the requirement to have an accurate time reference.

One or more embodiments of the invention provide for methods ofdetecting cheating in online game activities, and/or detecting cheatingand/or inaccurate reporting in activities related to news gathering,reporting, current events, disaster relief, stock market trading,forex/equity trading, real estate transactions, financial transactions,and other such activities, without limitation, the activities associatedwith one or more events, which methods can comprise one or morecombinations of analyzing latency in individual and cohort responses,machine-trained analysis of user cohort behavior, and analysis ofcrowdsourcing information derived from the same or other cohorts.

Some embodiments of the invention provide generally for analyzingcrowdsourcing information to analyze events that are the subject of theuser responses, for utilizing recursive queries to prompt secondarycrowdsource responses, and additionally or alternatively usingprobabilistic analysis of multiple data sources to form synchronizationacross delivery paths, locales and modes.

One or more embodiments provide for synchronizing input from a largenumber of devices to ascertain content, time, and time differences fromvarying external stimuli. In one embodiment a viewer can see an event attime, t(e) (or t.sub.event), a viewer can respond to an event at a timet(r) (or t.sub.response), and the system and/or method can process theviewers' responses at time t(p) (or t.sub.process). In embodiments, aplurality of data sources can be used in relation to real-time eventtranscription, cheating detection and compensation, and/or synchronizinguser engagement (such as, for example, without limitation: wirelesssignal, GPS, device accelerometer, absolute time, microphone input,logical location, delivery system base delay, event time, calculatedevent time, user event-based choice, and user choice time), any or allof which may be made part of the data processing and informationtransformation steps. In one or more embodiments, a delay estimation canbe made by: (i) user-stated mode, (ii) geo-location, (iii) devicemetadata including type of device, connection method,carrier/connectivity provider, etc. and/or (iv) comparing response timeand accuracy to other media/modes. The system and methods can determine,calculate and/or generate latency analysis based on t(r), on t(e), or ona difference between t(r) and t(e), which can be performed in referenceto a standard and precise clock time. Various embodiments can furtherprovide for advanced latency accounting for typical stimuli elicitingresponses and/or weighting the probability of a particular stimuli toelicit a response. An embodiment can provide, too, for identifyingcontrol groups that exhibit less or more appropriate latency.

One or more embodiments provide for identifying fraud, cheating, orother inaccurate input among many users reacting to stimuli via aplurality of media and methods. An embodiment, for example, can provideand utilize in the processing method(s), without limitation, at leastone or more of the following sub methods and/or information sources:randomized questions; “honeypot” testing for other transmission modes,control cohorts (such as, for example, without limitation, knownnon-cheaters and/or known cheaters); aggregate and calculate averageresponse times, providing normal min-max ranges for response boundary toidentify cheating; calculating average correct response percentagenorms; using geo-location to define cohort for comparative norm;checking response time and accuracy correlation with other transmissionmodes; and/or utilizing accuracy measure(s) based on probabilities andsuccesses, e.g., A(i)=average[p(t)*Si(k, t). One or more embodiments canutilize one or more sources of data, such as, without limitation, datafrom the group of data sources comprising global positioning system(GPS) data, device accelerometer data, microphone input data, logicallocation data, content channel data, delivery system base delay data,event time data, calculated event display time data, user event-basedchoice data, and user choice time data. An embodiment can furtheridentify control groups with better latency and/or accuracy.

One or more embodiments provide for detecting and synchronizing inputtiming with neighboring devices by a variety of methods, including,without limitation, WiFi, WiMax, Bluetooth®, NFC, and/or similarmethods. An embodiment can have each device emit specific sound (withinand/or outside the audio band) detected by other devices' availablesensors. Another or the same embodiment can use input timing or audiocontent recognition to determine which method and media is being used tobroadcast the event to the viewers, and to further aid in the detectionof cheating attempts. An embodiment can use input timing to identifycontrol groups with better latency.

One or more embodiments can be used to differentiate sensor readingsbetween different entities, for example if a temperature sensor in theroom records the body temperature of a human, data from other sensorscan be used to associate and validate this reading with a particularperson. It should be noted that the same principle can apply to otherliving creatures as well as to other machines and other objects.

One or more embodiments can be used to ascertain that sensor readingsare not corrupted by inadvertent readings from multiple objects, forexample if there are two people in a room and a human body temperaturesensor erroneously takes readings from both of those people. It shouldbe well understood that the same principles can apply to blood pressuresensors, heart rate sensors, and so on.

One or more embodiments can include using machine learning, heuristics,pattern-matching, game-theory, and more in the processing method(s).

An embodiment can further provide for determining absolute time (andnature) of stimuli without physical presence at location(s) of stimuli.This can further comprise, without limitation, one or more of:displaying different stimuli to different groups, then correlatingresponses to predict what happened where and when; recursively using ofone set of responses to generate additional query/stimuli; processingbeing automated within the system; and using heuristics, comb filters,auto-correlation, statistical algorithms, machine learning algorithms,and/or multi-agent systems, inter alia, without limitation, in order toimprove the precision of the derived data. The processing in at leastone embodiment can one or more of: feed other stimuli of similar eventsas additional signals (past or present); dynamically change decisiontree (represents options for events that have taken and/or are takingplace, and for potential future events), wherein crowd-sourcingresponses can be used to prune decision-tree branches; compareindividual replies to group to detect cheating/fraud; produce one ormore transcripts of one or more events; and/or identify one or morecontrol groups with better accuracy and/or latency.

In embodiments, a reliability rating (e.g. the reliability ratingsdiscussed above in connection with FIG. 22 ) may be used to determinewhether one or more IoT devices (e.g., connected consumer devices,connected enterprise devices, and/or connected industrial devices) isdefective and/or partially defective, and/or if an emergency is takingplace. In embodiment, an individual sensor device itself may beconsidered an IoT device. In embodiments, the reliability of an IoTdevice may be determined based, at least in part, on the reliability ofone or more sensor devices that may be associated with the IoT device.In embodiments, reliability may be determined for an individual sensorwhich may itself be considered an IoT device. In embodiments, eachsensor device of a plurality of sensor devices associated with the IoTdevice may be associated with sensor information. In embodiments, forexample, a computing device (e.g. computing device 2702) may be incommunication with the plurality of sensors, either directly orindirectly. In embodiments, the computing device may obtain and storesensor information from each sensor device of the plurality of sensordevices operatively connected to the computing device. In embodiments,the sensor information may include information that may assist thecomputing device in determining whether the TOT device or individualsensor device is at least partially defective. In embodiments, thesensor information may include one or more of the following: sensoridentification information (e.g. data unique to each sensor device);sensor type (e.g. light sensor, temperature sensor, acetometer, etc.),location information (e.g. data which may indicate a respective sensordevice's location); and/or data range information (e.g., data which mayindicate an acceptable range of values for a respective sensor device tooutput), to name a few.

The sensor information, in embodiments, may be used to separate theplurality of sensor devices into one or more groups. In embodiments, thecomputing device may determine whether a sensor device and/or an IoTdevice is defective, or partially defective based at least in part onthe group it is in. For example, the computing device, in embodiments,may select one or more sensor devices and group said sensor devices intoa first group of sensor devices. In embodiments, the computing devicemay select the first group of sensor devices based on the sensorinformation. For example, in embodiments, the computing device may groupsensor devices in a similar location in the same group. In embodiments,the computing device may group sensors of the same type in the samegroup. In embodiments, the computing device may group sensors withsimilar data range information in the same group, to name a few.

In embodiments, the sensor information may be used to correlate sensordata from a plurality of IoT devices (and other data sources) in orderto identify individual persons (or other creatures, or other objects)according to each's individual “signature” of sensor readings, with saidsignature being computed from the patterns of each individual sensorreadings and the combination of a plurality of patterns (andrelationships thereof) between patterns

In embodiments, to determine whether a sensor device and/or IoT deviceis defective, or partially defective, the computing device may receivesensor feedback information. In embodiments, the plurality of IoTdevices and/or sensor devices may obtain, generate, and/or transmitsensor feedback information (e.g. data provided by the sensor, orassociated IoT device and received by the computing device). The sensorfeedback information may include one or more of the following:identification information (e.g. data that identifies the source of thesensor feedback information), feedback data (e.g. data provided by thesensor device to the computing device in response to stimuli), and/or atimestamp (e.g. data indicating the time at which the feedback data wasobtained by the sensor device), to name a few.

For example, in embodiments, an IoT device (e.g. First IoT Device4304-1) may include one or more of the following: one or moremicrocontrollers (e.g. microcontroller(s) 4304-1A), one or more sensordevices (e.g. sensor(s) 4304-1B), an analog to digital converter (“ADC”)(e.g. optional ADC 4304-1C), a network connection interface (e.g.network connection interface 4304-1D), and/or a power source (e.g. powersource 4304-1E). In embodiments, the IoT device may include athermometer as a sensor device, which may be monitoring the ambienttemperature of the IoT device. Data representing temperature sensed bythe thermometer, in embodiments, may be transmitted to the ADC where thedata is converted from an analog stream to a digital stream. Onceconverted, in embodiments, the digital data representing the temperaturemay be transmitted to the one or more microcontrollers where the digitaldata may be processed by the one or more microcontrollers into sensorfeedback information. The processing, in embodiments, may include addinga timestamp, which may indicate the time and/or date at which thetemperature was sensed. Once the data representing the ambienttemperature of the IoT device is processed, in embodiments, the IoTdevice may transmit the sensor feedback information to the computingdevice. In embodiments, an individual sensor may be the IoT device andmay provide the feedback information directly to the computing device.In embodiments, the sensor device may include the ADC converter and thenetwork interface to allow for directed communication. In embodiments,the sensor device may include the microcontroller as well to process theraw output of the sensor device and provide the feedback information. Inembodiments, the raw output of the sensor device may be send to thecomputing device and processed at the computing device to provide thefeedback information.

In embodiments, when sensor feedback information is received at thecomputing device, the computing device may generate a second timestampindicating a time and/or date at which the sensor feedback informationwas received by the computing device. The sensor feedback information,in embodiments, may be processed by the computing device. The processedsensor feedback information may result in the generation and storing ofsensor variance information and/or sensor deviation information. Thisprocess may be completed for each IoT device and/or sensor device of thefirst group of devices simultaneously, as sensor feedback information isreceived, and/or at predetermined intervals, to name a few.

In embodiments, the computing device may also obtain and store weatherinformation associated with the respective location of the IoT deviceand/or sensor devices based on the location information associatedtherewith. The weather information, which may be obtained from a thirdparty (e.g. third party database 4310), may be obtained at regularintervals and/or as the sensor feedback information is received, to namea few. The weather information, in embodiments, may include, forexample, one or more of the following: temperature, atmosphericpressure, wind, humidity, precipitation and/or cloudiness, to name afew. In embodiment, one or more sensors may provide informationregarding temperature, atmospheric pressure, wind, humidity,precipitation and/or cloudiness, to name a few.

The respective sensor variance information, respective sensor deviationinformation, respective weather information, respective sensorinformation, and/or respective previous reliability rating associatedwith each respective sensor device may, in embodiments, be used whollyor in part to determine a current reliability rating of each sensordevice. In embodiments, the current reliability rating may be assignedand stored by the computing device.

A respective current reliability rating, respective sensor varianceinformation, respective sensor deviation information, respective weatherinformation, respective sensor information, and/or respective previousreliability rating associated with each respective sensor device may, inembodiments, be used to determine whether the respective IoT deviceand/or sensor device is defective or partially defective.

In embodiments, for example, where the IoT device is defective orpartially defective, the computing device may cure the defect and/orreport the defect. In embodiments, the computing device may flag the IoTdevice as defective, indicating that the computing device should“ignore” the bad sensor data (feedback information) provided from thedefective IoT device. The computing device, in embodiments, may alsogenerate and/or transmit one or more purchase orders to one or morevendors to purchase one or more parts necessary to cure the defect inthe defective IoT device and/or replace the defective IoT device ofsensor device. A more detailed explanation of an exemplary process fordetermining whether an IoT device is defective or at least partiallydefective is provided below in connection with the description of FIGS.44A-44G, the description of which applying herein.

In embodiments, a process for determining whether an IoT device isdefective or partially defective is described in connection with FIG.44A, which may begin at step S4402. At step S4402, in embodiments, aplurality of sensor devices may be provided. In embodiments, theplurality of sensor devices may be included in a plurality of IoTdevices (e.g. plurality of IoT devices 4302). The plurality of sensordevices, in embodiments, may be operatively connected via a network to acomputing device.

Referring to FIG. 43A, the plurality of sensor devices (e.g., theplurality of IoT devices 4302) may be operatively connected to thecomputing device 2702 via network 100. In embodiments, each of theplurality of IoT devices 4302 may be associated with respective sensorinformation. In embodiments, the computing device 2702 may beoperatively connected to one or more electronic devices respectivelyassociated with one or more users. In embodiments, each sensor devicemay be considered an IoT device and may be associated with respectivesensor information. For each IoT device, in embodiments, the sensorinformation may include data representing one or more of the following:(1) sensor identification information; (2) location information; (3)data range information; (4) sensor specification information; (5) sensorrunning time information; (6) sensor proximity information; (7)historical reliability rating information; (8) historical sensorvariance information; (9) historical sensor deviation information; (10)hierarchical information; and/or (11) historical defective reportinformation, to name a few. In embodiments, the sensor identificationinformation may be information unique to the respective IoT Device or arespective sensor device that may identify the IoT device or therespective sensor device. For example, the sensor identificationinformation may include one or more of the following: a UniversallyUnique Identifier (UUID), a Globally Unique Identifier (GUID), aPersistent Unique Identifier (PUID), a series of letters, numbers,and/or symbols, a unique name, electronic identification (may be similarto electronic identification described above in connection with FIG. 22, the description of which applying herein), connection information (maybe similar to connection information described above in connection withFIG. 22 , the description of which applying herein), and/or a sensortype, to name a few. For example, thermometer 4344 may be associatedwith the following sensor information:

-   -   Thermometer 4344 Sensor Identification Information    -   Unique Name - First Floor Thermometer    -   Sensor Type - Thermometer

In embodiments, location information may be information indicating thelocation of the respective IoT device or sensor device. For example, thelocation information may include one or more of the following: latitude,longitude, altitude, address, city, state, and/or zip code, to name afew. In embodiments, the location information described herein may besimilar to location information 2802C, the description of which applyingherein.

In embodiments, data range information may be information indicating anacceptable and/or expected range of feedback data the computing device2702 is to receive from the IoT device. For example, the data rangeinformation may include one or more of the following: a maximum sensorvalue indicating an upper threshold of sensor feedback data associatedwith the IoT device or sensor device; a minimum sensor value indicatinga lower threshold of sensor feedback data associated with the IoT deviceor sensor device, a data range over a period of time, a sensor variancerange, a sensor deviation range, and/or an emergency thresholdindicating that data received that passes the emergency thresholdindicates an emergency, to name a few. For example, referring to FIG.43D, an IoT device or specific sensor may have a data range representedby acceptable range 4358.

Referring to FIG. 43A, in embodiments, sensor specification informationmay be information indicating one or more of the following: the sensortype, the measurement the sensor detects, the resolution of theaforementioned measurement the sensor detects, a data range of valuesthe sensor is capable of measuring, an accuracy rating of the sensor, auseful life of the sensor device, and/or one or more parts associatedwith the sensor device, to name a few. In embodiments, the one or moreparts associated with the sensor device may include part numbers, vendorinformation, and/or a useful life of each part, to name a few. Forexample, Accelerometer 4312 may have the following exemplary sensorspecification information

Accelerometer 4312 Sensor Specification Information Sensor Type -Accelerometer Measurement - Meters per Second Squared Data Range -0-1,000 Accuracy - +/−2% Sensor Spec. - Useful Life 3.2 Years

Exemplary specification information for an exemplary sensor device isshown in connection with FIG. 43E. Referring to FIG. 43E, Sensor DeviceA may be associated with Sensor Device A Specification Information 4360.In embodiments, Specification Information 4360 may include one or moreof the following: a listing of parts by part number, a useful life ofeach part, one or more vendors associated with each part, a contact forthe vendor (e.g. a person, company, etc.), an e-mail address associatedwith the vendor and/or contact, a phone number associated with thevendor and/or contact, a fax number with the vendor and/or contact,information indicating whether a purchase order is available for thepart, a purchase order for the part, price information associated withthe part, and/or information indicating whether the part is in stock, toname a few. For example, Sensor Device A may include three parts—Partnumber 124A; Part number 672C; and Part number 249E. Continuing theexample, Part number 124A, as shown in FIG. 43E, has a useful life of3.4 years and is sold by Vendor 1. Vendor 1's contact is Alice and here-mail address is Alice@A.com. Additionally, Part number 124A has apurchase order on file—PO1. Continuing the example, Part number 672C, asshown in FIG. 43E, has a useful life of 1.2 years and is sold by Vendor2. Vendor 2's contact is Bob and his e-mail address is Bob@B.com.Additionally, Part number 672C has a purchase order on file—PO2.Continuing the example, Part number 249E, as shown in FIG. 43E, has auseful life of 5.1 years and is sold by Vendor 1. Additionally, Partnumber 249E has a purchase order on file—PO1. In embodiments, thepurchase orders on file may be stored by memory device 4308 which may beoperatively connected to the computing device 2702. In embodiments, eachpart may have multiple vendors, multiple contacts, no vendors, nocontacts, multiple purchase order forms, and/or no purchase order forms,to name a few.

In embodiments, sensor running time information may be informationindicating how long the IoT device, sensor device, and/or one or morepart(s) thereof have been powered on and running. For example, thesensor running time information may include one or more of thefollowing: the purchase date, the time the device and/or part was puton-line, interruptions within the running time, and/or the useful lifeof the device and/or part(s), to name a few.

In embodiments, sensor proximity information may be informationindicating one or more IoT devices and/or sensor devices within adistance of the IoT device and/or sensor device. For example, IoT device1 may have sensor proximity information indicating that IoT device 3 andIoT device 8 are within 10 meters of IoT device 1. Sensor proximityinformation may be similar to the proximity information described abovein connection with FIG. 22 , the description of which applying herein.

In embodiments, historical reliability rating information may beinformation indicating one or more reliability ratings that werepreviously assigned to the IoT device or sensor device. For example, thehistorical reliability rating information may include one or more of thefollowing: one or more reliability ratings associated with the IoTdevice or sensor device and/or the most recent reliability ratingassociated with the IoT device or sensor device, to name a few. Thereliability ratings discussed herein may be similar to the reliabilityratings described in connection with FIG. 22 , the description of whichapplying herein.

In embodiments, historical sensor variance information may beinformation indicating a variance of the data received from the IoTdevice or sensor device. For example, the historical sensor varianceinformation may include one or more of the following: one or more sensorvariance values associated with the IoT device or sensor device and/orthe most sensor variance value associated with the IoT device, to name afew.

In embodiments, historical sensor deviation information may beinformation indicating a deviation of the data received from the IoTdevice or sensor device. For example, the historical sensor deviationinformation may include one or more of the following: one or more sensordeviation values associated with the IoT device and/or the most sensordeviation value associated with the IoT device, to name a few.

In embodiments, hierarchical information may be information used togroup a plurality of sensor devices and/or IoT devices. For example, aplurality of IoT Devices may be monitoring an apartment building withfive floors. Continuing the example, the apartment building may receivepower at a main circuit breaker, which may be monitored by a first IoTDevice. The main circuit breaker may be operatively connected to acircuit breaker for each floor—a first floor circuit breaker, a secondfloor circuit breaker, a third floor circuit breaker, a fourth floorcircuit breaker, and a fifth floor circuit breaker. Each circuit breakermay be monitored by a respective IoT Device (a second IoT Device, athird IoT Device, a fourth floor IoT Device, a fifth IoT Device and asixth IoT Device). Continuing the example, each floor may also bemonitored by an IoT Device (a seventh IoT Device, an eighth IoT Device,a ninth floor IoT Device, a tenth IoT Device and an eleventh IoTDevice). The electricity, in this example, may flow through the maincircuit breaker to each floor. In this example, the hierarchicalinformation may indicate the following information:

TABLE 26 IoT Device Hierarchical Information First IoT Device MainCircuit Breaker Second IoT Device First Floor Circuit Breaker Third IoTDevice Second Floor Circuit Breaker Fourth IoT Device Third FloorCircuit Breaker Fifth IoT Device Fourth Floor Circuit Breaker Sixth IoTDevice Fifth Floor Circuit Breaker Seventh IoT Device First FloorApartment Eighth IoT Device Second Floor Apartment Ninth IoT DeviceThird Floor Apartment Tenth IoT Device Fourth Floor Apartment EleventhIoT Device Fifth Floor Apartment

In embodiments, historical defective report information may beinformation indicating one or more previous malfunctions of the IoTdevice or sensor device and/or one or more previous malfunctions of theIoT devices or sensor devices in proximity of the IoT device or sensor.For example, the historical defective report information may include oneor more of the following: one or more reports of malfunctions of the IoTdevice, one or more malfunctions of IoT devices within a proximity ofthe IoT device, and/or whether the IoT device is currently at leastpartially defective, to name a few.

In embodiments, respective sensor information may be received from eachrespective IoT device or sensor device of the plurality of IoT devices4302. In embodiments, respective sensor information may be manuallyinputted by an administrator. In embodiments, respective sensorinformation may be received from a third party.

Referring to FIG. 43B, as stated above, the plurality of IoT Devices4302, in embodiments, may be a plurality of sensor devices. Theplurality of IoT Devices 4302 may be and/or may include one or more ofthe following sensors from the following non-exhaustive list of sensors:(1) location sensor 4332, (2) accelerometer 4312, (3) altimeter 4314,(4) gyrometer 4316, (5) magnetometer 4318, (6) device motion sensor4322, (7) proximity sensor 4324, (8) light sensor 4326, (9) pressuresensor 4320, (10) temperature sensor 4328, (11) air bubble detector4330, (12) touch screen sensor 4334, (13) finger print sensor 4336, (14)barcode/QR code sensor 4338, (15) barometer 4340, (16) heart rate sensor4342, (17) thermometer 4344, (18) air humidity sensor 4346, (19) Geigercounter 4348, (20) torque sensor 4350, (21) MEMS Sensor 4352, (22) Piezofilm sensor 4354, (23) humidity sensor 4356; (24) blood pressuresensor(s); (25) microphone(s) (and/or other acoustic sensors); (26)brain wave sensor(s) (e.g. EEG); (27) heart wave monitor(s) (e.g., EKG);(28) radar sensor(s); (28) chemical composition sensors (e.g. PCR and/orany form of chromatography) (29) electrical current sensors; (30)electrical voltage sensors; (31) electrical capacitance or inductionsensors; (32) neurological sensors; (33) pH sensors; (34) salinitysensors; (35) biological assay sensors and/or (36) LIDAR sensor(s), toname a few.

Referring to FIG. 44A, step S4402 may further include a step ofproviding a reliability rating database. The reliability ratingdatabase, in embodiments, may be operatively connected to the computingdevice 2702. The reliability rating database, in embodiments, may storethe above described historical reliability rating information and/orcurrent reliability rating information (which may be determined and/orassigned in connection with step 4524, the description of which applyingherein), to name a few.

In embodiments, step S4402 may further include a step of providing atleast one hierarchical network model. Each of the at least onehierarchical network models may be associated with at least oneiterative algorithm. The at least one iterative algorithm, inembodiments, may be used to derive a hierarchical network model, whichmay be used in whole or in part by the computing device 2702 to selectone or more groups of sensor devices (e.g. first group of IoT Devices4304). For example, the computing device 2702 may apply the at least oneiterative algorithm to each sensor device's respective sensorinformation, the results of which may assist in the, or lead to, aselection of the first group of IoT Devices 4304. The at least onehierarchical network model and/or the associated at least one iterativealgorithm may be stored by the computing device 2702.

In embodiments, a process for determining whether an IOT device orsensor device is at least partially defective may continue with stepS4404. At step S4404, the provided sensor information may be stored bythe computing device 2702. For example, the provided sensor informationmay be stored in memory 2702-3. Referring to FIG. 43A, the computingdevice 2702 may be operatively connected to memory device 4308. Thesensor information, when provided, may be stored by the computing device2702 in the memory device 4308. Memory device 4308, in embodiments, mayinclude one or more types of storage mediums such as any volatile ornon-volatile memory, or any removable or non-removable memoryimplemented in any suitable manner to store data for the computingdevice 2702. For example, the sensor information may be stored usingcomputer-readable instructions, data structures, and/or program systems.Various types of storage/memory may include, but are not limited to,hard drives, solid state drives, flash memory, permanent memory (e.g.,ROM), electronically erasable programmable read-only memory (“EEPROM”),CD-ROM, digital versatile disk (“DVD”) or other optical storage medium,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, RAID storage systems, or any other storagetype, or any combination thereof. Furthermore, memory device 4308 may beimplemented as computer-readable storage media (“CRSM”), which may beany available physical media accessible by the computing device 2702 toexecute one or more instructions stored within memory device 4308. Inembodiments, memory 2702-3 of the computing device 2702 may be similarto memory device 4308, the description of which applying herein.

Referring to FIG. 44A, in embodiments, the process for determiningwhether an IOT device or sensor device is at least partially defectivemay continue with step S4406. At step S4406, the sensor information maybe accessed by a sensor module of the computing device 4506. Forexample, the sensor module may obtain the sensor information stored inmemory device 4308 and/or memory 2702-3. For example, the sensor modulemay receive the sensor information stored in memory device 4308 and/ormemory 2702-3. Referring to FIG. 43A, the computing device 2702 mayinclude, in embodiments, a sensor module 4306. The sensor module 4306,in embodiments, may include one or more of the following:

processor(s) 4306-1, memory, and/or network connection circuitry, toname a few. The sensor module 4606 may be implemented as combinations ofsoftware, hardware, and/or firmware. For example, the sensor module 4606software may include instructions in one or more suitable languages andmay be stored on one or more computer readable storage media which canbe accessed and/or executed by one or more processors (e.g., processor4306-1, processor(s) 2702-2, to name a few). The sensor module 4606 mayform particular elements that perform respective functions related tothe methods and processes described in further detail below according toembodiments of the present invention. In embodiments, variousalgorithmic processes may be implemented for the purpose of processingsensor information and/or sensor feedback information.

Referring to FIG. 44A, in embodiments, the process for determiningwhether an IOT device or sensor device is at least partially defectivemay continue with step S4408. At step S4408, a first group of sensordevices is selected from the plurality of sensor devices. Inembodiments, the first group of sensor devices and/or IoT devices may beselected by one or more of the sensor module 4306 and/or the computingdevice 2702. For example, the sensor module 4306 may extract respectivelocation information and respective sensor type information fromrespective sensor information associated with each IoT device or sensordevice of the plurality of IoT devices 4302 or sensor devices. Thesensor module 4306, in embodiments, may use the respective locationinformation to select a group of IoT devices within a distance of aparticular location. For example, the group of IoT devices or sensordevices may be within one or more of the following: a predeterminedradius of a particular location, a radius of a particular location, adistance from a particular location, an area, a building, a city, atown, a state, and/or a country to name a few. The sensor module 4306may then determine the first group of IoT devices 4304 by selecting asecond group of IoT devices based on the respective type of each sensordevice. Thus, in this example, the first group of IoT devices 4304 orsensor devices may be IoT devices or sensor devices of the plurality ofIoT devices 4302 that are of the same type of sensor and are within adistance of a particular location. In embodiments, the first group ofIoT devices 4304 or sensor devices may be selected based on one or moreof the following: respective location information, respective sensorinformation, respective sensor identification information, respectivesensor type, sensor specification information, respective hierarchicalinformation, hierarchical information, and/or current weatherconditions, to name a few.

Referring to FIG. 43C, in embodiments the first group of IoT devices4304 or sensor devices may include: first IoT device 4304-1, second IoTdevice 4304-2, and third IoT device 4304-3. In embodiments, first IoTdevice 4304-1 may include one or more of the following:microcontroller(s) 4304-1A, sensor(s) 4304-1B, (optional) analog todigital converter 4304-1C, network connection interface 4304-1D, and/orpower source 4304-1E, to name a few. In embodiments, second IoT device4304-2 may include one or more of the following: microcontroller(s)4304-2A, sensor(s) 4304-2B, (optional) analog to digital converter4304-2C, network connection interface 4304-2D, and/or power source4304-2E, to name a few. In embodiments, third IoT device 4304-3 mayinclude one or more of the following:

microcontroller(s) 4304-3A, sensor(s) 4304-3B, (optional) analog todigital converter 4304-3C, network connection interface 4304-3D, and/orpower source 4304-3E, to name a few.

In embodiments, microcontroller(s) 4304-1A, microcontroller(s) 4304-2A,and/or microcontroller(s) 4304-3A may include one or more of thefollowing: a central processing unit (CPU), random access memory (RAM),input circuitry, and/or output circuitry, to name a few. In embodiments,sensor(s) 4304-1B, sensor(s) 4304-2B, and sensor(s) 4304-3B may besimilar to one or more of the plurality of IoT devices 4302 describedabove in connection with FIG. 43B, the description of which applyingherein.

In embodiments, network connection interface 4304-1D, network connectioninterface 4304-2D, network connection 4304-3D, and network connectioninterface 2702-1 may include any circuitry allowing or enabling one ormore components of the first IoT device 4304-1, second IoT device4304-2, third IoT device 4304-3, and/or the computing device 2702respectively to communicate with one another, with computing device2702, and/or with one or more additional devices, servers, and/orsystems. As an illustrative example, data retrieved from the one or moreIoT devices may be transmitted over a network 100, such as the Internet,to computing device 2702 using any number of communications protocols.For example, network(s) 100 may be accessed using Transfer ControlProtocol and Internet Protocol (“TCP/IP”) (e.g., any of the protocolsused in each of the TCP/IP layers), Hypertext Transfer Protocol(“HTTP”), WebRTC, SIP, and wireless application protocol (“WAP”), aresome of the various types of protocols that may be used to facilitatecommunications between first IoT device 4304-1, second IoT device4304-2, and third IoT device 4304-3, and/or the computing device 2702.In some embodiments, first IoT device 4304-1, second IoT device 4304-2,third IoT device 4304-3, and/or the computing device 2702 maycommunicate with one another via a web browser using HTTP. Variousadditional communication protocols may be used to facilitatecommunications between first IoT device 4304-1, second IoT device4304-2, and third IoT device 4304-3, and/or the computing device 2702,include the following non-exhaustive list, Wi-Fi (e.g., 802.11protocol), LiFi, Bluetooth®, radio frequency systems (e.g., 900 MHz, 1.4GHz, and 5.6 GHz communication systems), optical communications,cellular networks (e.g., GSM, AMPS, GPRS, CDMA, EV-DO, EDGE, 3GSM, DECT,IS-136/TDMA, iDen, LTE, 4G, 5G, 6G or any other suitable cellularnetwork protocol), infrared, BitTorrent, FTP, RTP, RTSP, SSH, and/orVOIP.

In embodiments, network connection interface 4304-1D, network connectioninterface 4304-2D, network connection 4304-3D, and network connectioninterface 2702-1 may use any communications protocol, such as any of thepreviously mentioned exemplary communications protocols. In someembodiments first IoT device 4304-1, second IoT device 4304-2, third IoTdevice 4304-3, and/or the computing device 2702 may include one or moreantennas to facilitate wireless communications with a network usingvarious wireless technologies (e.g., Wi-Fi, Bluetooth®, radiofrequency,etc.). In yet another embodiment, first IoT device 4304-1, second IoTdevice 4304-2, third IoT device 4304-3, and/or the computing device 2702may include one or more universal serial bus (“USB”) ports, one or moreEthernet or broadband ports, and/or any other type of hardwire accessport so that network connection interface 4304-1D, network connectioninterface 4304-2D, network connection 4304-3D, and network connectioninterface 2702-1 allows first IoT device 4304-1, second IoT device4304-2, third IoT device 4304-3, and/or the computing device 2702respectively to communicate with one another or with one or morecommunications networks.

In embodiments, power source 4304-1E, power source 4304-2E, and powersource 4304-3E may be a device capable of providing power to first IoTdevice 4304-1, second IoT device 4304-2, and third IoT device 4304-3respectively.

Referring to FIG. 44A, in embodiments, the process for determiningwhether an IOT device is at least partially defective may continue withstep S4410. At step S4410, the sensor module 4306 and/or the computingdevice 2702 may obtain sensor feedback information from one or moresensor devices of the first group of sensor devices (e.g. the firstgroup of IoT devices 4304). For example, the first group of IoT devices4304 may be three torque sensors within a turbine—e.g. a first torquesensor, a second torque sensor, a third torque sensor. Once the threetorque sensors are powered on and operatively connected to the computingdevice 2702 via network 100, each torque sensor of the first group ofIoT devices 4304 may begin sensing torque. Continuing the example, datarepresenting the sensed torque may be processed, resulting in sensorfeedback information representing, at least in part, the sensed torque.

The sensor feedback information, in embodiments, may be transmitted bythe first group of IoT devices 4304 to the sensor module 4306 via thecomputing device 2702 over network 100. In embodiments, the sensorfeedback information may be transmitted as the sensed torque data isprocessed into the sensor feedback information and/or transmitted atregular intervals, to name a few. In embodiments, the sensor feedbackinformation may be stored locally by the first group of IoT devices4304, transmitting the sensor feedback information only upon request(e.g. by the computing device 2702).

In embodiments, the sensor feedback information, for each IoT device,may include one or more of the following: (1) identificationinformation, (2) feedback data, (3) a timestamp, (4) connectioninformation, (5) a current reliability rating, and/or (6) deviceinformation, to name a few.

In embodiments, identification information may refer to informationassociated with the IoT device providing the sensor feedbackinformation. The identification information, in embodiments, may enablethe computing device 2702 to identify to IoT device which provided thesensor feedback information. In embodiments, the identificationinformation may include one or more of the following: respective sensoridentification information, a Universally Unique Identifier (UUID), aGlobally Unique Identifier (GUID), a Persistent Unique Identifier(PUID), a series of letters, numbers, and/or symbols, a unique name,electronic identification (may be similar to electronic identificationdescribed above in connection with FIG. 22 , the description of whichapplying herein), connection information (may be similar to connectioninformation described above in connection with FIG. 22 , the descriptionof which applying herein), and/or a sensor type, to name a few.

In embodiments, feedback data may refer to the data which was sensed bythe IoT device or sensor device. For example, torque sensor 4350 mayprovide torque feedback data. In embodiments, the feedback data mayinclude data over a range of time and/or current data, to name a few.

In embodiments, the timestamp may refer to the time at which thefeedback data was sensed by the IoT device or sensor device. Thetimestamp may be specific to one or more data entries of the feedbackdata. For example, the feedback data may include data sensed over aperiod of time. Continuing the example, if an IoT device or sensordevice senses stimuli, generating data points between the times of 6:39AM and 7:32 AM, the timestamp included with the sensor feedbackinformation from the IoT device may indicate the time each data pointwas gathered by the IoT device or sensor device. If, for example, theIoT device gathers data at 6:39 AM, 6:52 AM, and 7:32 AM, the timestampmay indicate each of those times. In embodiments, the time stamp mayrefer to the time in which the sensor feedback information was sent tothe computing device 2702.

In embodiments, connection information may be similar to connectioninformation described above in connection with FIG. 22 , the descriptionof which applying herein. In embodiments the connection information maybe used to give context to the sensor feedback information. For example,a time lag may be detected and associated with a timestamp if the sensorfeedback information and the timestamp are not in line with sensorfeedback information received from other IoT devices within the samegroup of IoT devices.

In embodiments, a current reliability rating may refer to a reliabilityrating that was previously assigned to the IoT device or sensor deviceproviding the sensor feedback information. In embodiments, when areliability rating is determined and assigned, the computing device 2702may send a respective reliability rating to one or more of the IoTdevices of sensor device of the plurality of IoT devices 4302 or sensordevices. Once received, in embodiments, the respective IoT device maystore the respective reliability rating—transmitting the reliabilityrating with the sensor feedback information.

In embodiments, device information may refer to information regardingthe IoT device or sensor device that processed the sensor feedbackinformation. For example, the device information may include one or moreof the following: a diagnostic report of the IoT device and/or a batterylevel of the IoT device, to name a few.

In embodiments, the process for determining whether an IOT device is atleast partially defective may continue with step S4412. At step S4412,the sensor module 4306 and/or the computing device 2702 may generate asecond timestamp. The second timestamp, in embodiments, may indicate atime at which the sensor feedback information was obtained.

In embodiments, the process for determining whether an IOT device is atleast partially defective may continue with step S4414. At step S4414,the sensor feedback information and second timestamp are stored by thecomputing device 2702 and/or the sensor module 4306. In embodiments, thesensor feedback information and/or the second time stamp may be stored,separately and/or together, in one or more of the following: memory2702-3, memory of the sensor module 4306, and/or the memory device 4308,to name a few.

Referring to FIG. 43A, in embodiments, the computing device 2702 mayalso obtain additional information from one or more third partydatabase(s) 4310. In embodiments, the additional information may includeone or more of the following: weather information, vendor information(e.g. who the vendors are, whether the vendors are open, what parts arein stock, etc.), calendar information (the date, weekend, holidays,etc.), and/or IoT Device administrator information (e.g. who theadministrator is, what their calendar states, etc.), to name a few. Forexample, the computing device 2702 may take current weather conditionsinto account when determining whether at least one IoT device or sensordevice is at least partially defective. Continuing the example, thecomputing device 2702 may request the third party database 4310 forweather information at the time of the timestamp received with thesensor feedback information, at the location associated with thelocation data.

The request may be generated by the computing device 2702 andtransmitted to the third party database 4310 via network 100, which, inembodiments, may result in the third party database 4310 transmittingthe current weather conditions to the third party database 4310. Inembodiments, the computing device 2702 may obtain additional informationin substantially real time. For example, the computing device 2702 mayutilize a streaming HTTP protocol to receive data through and establisha streaming application programming interface (SAPI) with the thirdparty database 4310. Once the connection is established, the additionalinformation may be streamed, in substantially real time, from the thirdparty database 4310 to the computing device 2702 via network 100 and theSAPI.

Referring to FIG. 44A, in embodiments, the process for determiningwhether an IOT device is at least partially defective may continue withstep S4416. At step S4416, the sensor module 4306 and/or the computingdevice 2702 may generate sensor variance information for each sensordevice of the first group of sensor devices. In embodiments, the sensorvariance information may be used to determine one or more of: astability of the noise associated with the sensor feedback information,a systematic bias, a difference between noise forms, the reliability ofthe feedback data received, and/or an authenticity of the feedback datareceived, to name a few. To generate sensor variance information foreach sensor device and/or IoT device of the first group of IoT devices4304, the sensor module 4306 and/or the computing device 2702 may applyat least one of the following algorithms to the obtained feedback data:the two-sample variance equation, the Allan deviation, the modifiedAllan variance, the total variance equation, the Hadamard varianceequation, and/or the time variance equation, to name a few. For example,the Allan variance may be represented in the following equation:

${AVA{R^{2}(\tau)}} = {\frac{1}{2 \cdot ( {n - 1} )}{\sum\limits_{i}( {{y(\tau)}_{i + 1} - {y(\tau)}_{i}} )^{2}}}$

The above exemplary equation is the Allan Variance (AVAR) as a functionof averaging time. In embodiments, one or more algorithms (and/orincluding AVAR) may be utilized to generate sensor feedback information,in whole or in part. For example, one or more Euclidean Distancescalculations may be utilized with the AVAR to generate sensor feedbackinformation. In embodiments, the sensor variance information may bebased on at least one of the following: the sensor feedback information,the sensor information, the aforementioned additional information (e.g.weather information), the first timestamp (e.g. associated withgathering and/or transmitting of the sensor feedback information),and/or the second timestamp (e.g. associated with the receipt of thesensor feedback information, to name a few. In embodiments, one or moreEuclidean Distances calculations may be utilized without the AVAR togenerate sensor feedback information. In embodiments, the sensorvariance information may be based on at least one of the following: thesensor feedback information, the sensor information, the aforementionedadditional information (e.g. weather information), the first timestamp(e.g. associated with gathering and/or transmitting of the sensorfeedback information), and/or the second timestamp (e.g. associated withthe receipt of the sensor feedback information, to name a few.

In embodiments, the following is exemplary code for determining sensorvariance in accordance with exemplary embodiments of the presentinvention:

-   -   Exemplary Pseudocode

-   import numpy as np

-   import pandas as pd

-   from pandas import Series

-   from pandas import DataFrame

-   from sklearn.metrics.pairwise import

-   275uclidcan_distances

-   from sklearn.metrics.pairwise import

-   paireddi stances

-   from scipy spatial.distance import cdist

-   d={‘coll’: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,    16, 17, 18, 19, 20,21,22}, ‘col2’: [0, 1, 1,1,2, 1, 1, 2, 3, 2, 1,    1, 1, 1, 1, 2, 1, 1, 3,4, 1,2,3]}

-   df=pd.DataFrame(data=d)

-   d2={‘coll’: [0, 1,2], ‘col2’: [1,2,3]}

-   patch=pd.DataFrame(data=d2)

-   def orig(df, patch):

df[‘corr’]=np.nan

for i in range(df.shape[0]):

-   -   #select the df window with the same size of patch    -   window=df[i : i+patch.shape[0]]    -   #If window and patch have different shapes →

-   Break    -   if window shape[0]!=patch shape[0]:        -   break    -   else:        -   patch. reset index(inplace=True, drop=True)        -   window.reset index(inplace=True, drop=True)        -   ‘f[′c’rr′]=cdist(d‘[[′c’12′]], patc‘[[′c’12′’], ′euclid’an′)

return df

-   orig(df, patch)

As shown in the above, in embodiments, the sensor readings (d[col1]) andcorresponding timestamps (d[col2]) are analyzed for any patterns ofvariance.

In embodiments, the process for determining whether an IOT device is atleast partially defective may continue with step S4418. At step S4418,the sensor module 4306 and/or the computing device 2702 may generatesensor deviation information for each sensor device of the first groupof sensor devices. In embodiments, the sensor deviation information maybe used to determine one or more of: noise within a signal over time, asystematic bias, a difference between noise forms, the reliability ofthe feedback data received, and/or an authenticity of the feedback datareceived, to name a few. To generate sensor deviation information foreach sensor device and/or IoT device of the first group of IoT devices4304, the sensor module 4306 and/or the computing device 2702 may takethe square root of the sensor variance information. In embodiments, thesensor deviation information may be based on at least one of thefollowing: the sensor feedback information, the sensor information, theaforementioned additional information (e.g. weather information), thefirst timestamp (e.g. associated with gathering and/or transmitting ofthe sensor feedback information), and/or the second timestamp (e.g.associated with the receipt of the sensor feedback information), to namea few.

In embodiments, the process for determining whether an IOT device is atleast partially defective may continue with step S4420. At step S4420,the sensor variance information and the sensor deviation information foreach sensor device are both stored by the computing device 2702 and/orthe sensor module 4306. In embodiments, the sensor variance informationand/or the sensor deviation information for each sensor device may bestored separately and/or together, in one or more of the following:memory 2702-3, memory of the sensor module 4306, and/or the memorydevice 4308, to name a few.

In embodiments, the process for determining whether an IOT device is atleast partially defective may continue with step S4422. At step S4422,the computing device 2702 and/or sensor module 4306 may determine and/orassign a reliability rating to each sensor device of the first group ofsensor devices. In embodiments, the computing device 2702 may determinea reliability rating of an IoT device of a group of IoT devices basedone on or more of the following: respective sensor variance information,respective sensor deviation information, respective sensor information,the second timestamp, the aforementioned additional information (e.g.weather information), respective historical reliability ratings, sensorinformation associated with additional IoT devices of the group of IoTdevices, and/or historical reliability ratings associated with theadditional IoT devices of the group of IoT devices, to name a few.

For example, the computing device 2702 may obtain sensor feedbackinformation from the first group of IoT devices 4304. The first group ofIoT devices, for the purposes of the example, includes a First Device, aSecond Device, and a Third Device. Continuing the example, once thecomputing device 2702 has generated sensor variation information andsensor deviation information for the First Device, Second Device, andThird Device, the computing device 2702 may determine that the SecondDevice's sensor variance information and feedback data are outside anexpected range. The computing device 2702 may also determine that theFirst Device and the Third Device are associated with feedback data,sensor variation information, and sensor deviation information that areeach respectively within an expected respective range. Continuing theexample, because weather may affect the sensor variance information, thecomputing device 2702 may obtain weather information for the area inwhich the first group of IoT devices 4304 are located. In this example,the computing device may determine that the weather informationindicates that weather is not the cause of the sensor varianceinformation that is outside an expected range. The computing device2702, in embodiments, once it is determined that there are values out ofrange that are not explained by the First IoT Device, the Third IoTDevice, or the weather, may obtain historical reliability ratings foreach of the First IoT Device, the Second IoT Device, and the Third IoTDevice. Continuing the example, each of the First, Second, and Third IoTDevices may historically have reliable reliability ratings. Concludingthe example, given the disclosure in this example, the computing device2702 may determine that the Second IoT Device is unreliable and theFirst and Third IoT Devices are reliable—assigning each their respectivereliability rating.

The reliability rating, in embodiments, may be a binary rating (e.g.reliable or unreliable) and/or a graded scale (e.g. on a scale of 1-10where 1 is the most unreliable and 10 is the most reliable), to name afew. In embodiments, the determined reliability ratings may be storedseparately and/or together, in one or more of the following: memory2702-3, memory of the sensor module 4306, a reliability ratingsdatabase, and/or the memory device 4308, to name a few. In embodiments,a reliability rating that indicates an IoT device is unreliable does notnecessarily mean that the IoT device is at least partially defective,and vice versa.

In embodiments, after determining and assigning a reliability rating foreach of the IoT devices of the first group of IoT devices 4304, thecomputing device 2702 may generate a plurality of machine-readableinstructions, each of the plurality being for each IoT device of thefirst group of IoT Devices 4304. The machine-readable instructions, inembodiments, may be to store the respective, determined, and assignedreliability rating locally on the respective IoT device. Once generated,in embodiments, the plurality of machine-readable instructions may betransmitted by the computing device 2702 to each IoT device of the firstgroup of IoT devices 4304 via network 100. In embodiments, upon receiptof respective machine-readable instructions, the respective IoT devicemay store the respective reliability rating.

In embodiments, the computing device 2702 may generate a reliabilityreport. The reliability report may include one or more of the following:(1) current reliability ratings associated with the first group of IoTdevices 4304, (2) current reliability ratings associated with theplurality of IoT devices 4302, (3) historical reliability ratingsassociated with the first group of IoT devices 4304, (4) historicalreliability ratings associated with the plurality of IoT devices 4302,(5) the sensor feedback information associated with the first group ofIoT devices 4304, and/or (6) the sensor feedback information associatedwith the plurality of IoT devices 4302, to name a few. The reliabilityreport, in embodiments, may be transmitted by the computing device 2702to a device associated with at least one administrator of the computingdevice 2702 via network 100.

In embodiments, the process for determining whether an IOT device is atleast partially defective may continue with step S4424. At step S4424,the computing device 2702 and/or sensor module 4306 may determinewhether at least one sensor device of the first group of sensor devicesis at least partially defective. In embodiments, the computing device2702 may determine whether at least one sensor device of the first groupof sensor devices is at least partially defective based one on or moreof the following: respective reliability rating, respective sensorvariance information, respective sensor deviation information,respective sensor information, the second timestamp, the aforementionedadditional information (e.g. weather information), respective historicalreliability ratings, the first timestamp, sensor information associatedwith additional IoT devices of the group of IoT devices, and/orhistorical reliability ratings associated with the additional IoTdevices of the group of IoT devices, to name a few.

Continuing the example described in connection with step S4422, thecomputing device 2702 may determine that the Second Device's sensorvariance information and feedback data are outside an expected range.The computing device 2702 may also determine that the First Device andthe Third Device are associated with feedback data, sensor variationinformation, and sensor deviation information that are each respectivelywithin an expected respective range. The computing device 2702, inembodiments, once it is determined that there are values out of rangethat are not explained by the First IoT Device, the Third IoT Device,and/or the weather, may obtain historical reliability ratings for eachof the First IoT Device, the Second IoT Device, and the Third IoTDevice. Continuing the example, each of the First, Second, and Third IoTDevices may historically have reliable reliability ratings. Concludingthe example, given the disclosure in this example, the computing device2702 may determine that the Second IoT Device is at least partiallydefective.

In embodiments, a wide fluctuation in noise, indicated by sensorvariation information, may indicate the battery (e.g. power source4304-1E) of the IoT device (e.g. first IoT device 4304-1) is low and/orthe voltage from the battery is fluctuating. As another example, if therespective feedback data is a plurality of data points over a period oftime, and the slope of the curve connecting the data points is similarto that of the curve of data points associated with obtained feedbackdata from additional IoT devices of the same group of IoT devices, thecomputing device 2702 may determine that the IoT device is notcalibrated correctly. A more detailed explanation is described inconnection with FIG. 44C.

Referring to FIG. 44C, in embodiments, the process for determiningwhether at least one sensor device of the first group of sensor devicesis at least partially defective may begin at step S4430. At step S4430,the computing device 2702 and/or sensor module 4306 may compare thefirst respective sensor feedback information associated with the atleast one sensor device with first respective sensor informationassociated with the at least one sensor device. For the purposes of thefollowing example, the first group of IoT devices 4304 may includethermometer 4344, Thermometer A, and Thermometer B. Thermometer 4344,Thermometer A, and Thermometer B, for the purposes of this example, maybe located in an elevator machine room. Thermometer 4344, for example,may be located six feet off the floor of the elevator machine room.Thermometer A, for example, may be located on the floor of the elevatormachine room. Thermometer B, for example, may be located on the ceilingof the elevator machine room.

Continuing the example, the computing device 2702 may extract data fromthe stored sensor information of thermometer 4344 and data from theobtained sensor feedback data of thermometer 4344. As mentioned above,in embodiments, the sensor information for thermometer 4344 may includedata representing one or more of the following: (1) sensoridentification information; (2) location information; (3) data rangeinformation; (4) sensor specification information; (5) sensor runningtime information; (6) sensor proximity information; (7) historicalreliability rating information; (8) historical sensor varianceinformation; (9) historical sensor deviation information; and/or (10)historical defective report information, to name a few. Moreover, asmentioned above, in embodiments, the sensor information for thermometer4344 may include one or more of the following: (1) identificationinformation, (2) feedback data, (3) a timestamp, (4) connectioninformation, (5) a current reliability rating, and/or (6) deviceinformation, to name a few. Continuing the example, the computing device2702 may extract data range information from the sensor information ofthermometer 4344 and feedback data from the sensor feedback informationof thermometer 4344. For the purposes of this example, the exemplaryinformation extracted by the computing device 2702 is listed in thebelow table:

TABLE 27 Data Range Information Feedback Data Expected Range TimeTemperature Time 15.56° C.-32.22° C. 1:00 PM 134° F. 1:00 PM 15.56°C.-32.22° C. 2:00 PM  76° F. 2:00 PM 15.56° C.-32.22° C. 3:00 PM 121° F.3:00 PM 15.56° C.-32.22° C. 4:00 PM 143° F. 4:00 PM

As shown in the above exemplary table, the data range informationincludes an expected and/or acceptable temperature range over the courseof time. Because an elevator may be in service during an afternoon (e.g.between the hours of 1 PM and 4 PM), the expected range may be constant.However, in embodiments, the expected range may fluctuate based on oneor more conditions (e.g. planned service, high volume of usage during aperiod of time, low volume of usage, to name a few). In embodiments, thecomputing device 2702 may initially view the feedback data above as anindication of either a defective sensor or an emergency situation (e.g.a fire). In response, the computing device 2702 may process the data,looking for issues with either the feedback data or the expected range(e.g. different unit of measurement, different time, missing datapoints, to name a few). For example, the computing device 2702 maydetermine that the expected range is in a different unit of measurementthan the feedback data. Before determining thermometer 4344 is at leastpartially defective, the computing device 2702 may convert the units ofmeasurement such that the expected range and the feedback data arerepresented in the same unit of measurement.

Continuing the example, the computing device 2702 may, in embodiments,convert the expected range to Fahrenheit and/or convert the feedbackdata to Celsius. For the purposes of this example, the computing devicemay convert the expected range of 15.56° C.-32.22° C. to 60° F.-90° F.Once the expected range and the feedback data are represented in thesame unit of measurement, the computing device 2702 may determine that 3of the 4 data points between 1 PM and 4 PM are outside of the expectedrange. In embodiments, the computing device 2702 may compare the datapoints outside the expected range to predetermined thresholds. Forexample, in embodiments, a room that is on fire may have a temperatureof 100° F. at the floor level, 600° F. at eye level, and 1,500° F. atceiling level—depending on the size and shape of the room. Becausethermometer 4344 is located six feet off the floor of the elevatormachine room, the computing device 2702 may determine whether thetemperature is hot enough to indicate a fire—e.g. determine if thetemperature is 600° F. In embodiments, the predetermined threshold foran emergency situation (e.g. a fire) may be 400° F., allowing for afaster response to an emergency. For the purposes of this example,because the data points do not reach 400° F., the computing device 2702may determine that there is no emergency situation, and flag thethermometer 4344 for monitoring (see e.g. steps S4440-S4448, thedescription of which applying herein).

If, for example, the temperature is hot enough to indicate a fire, thecomputing device 2702 may generate and send a notification message (e.g.see steps S4426-S4428, the description of which applying herein) to oneor more of the following: an administrator of the computing device 2702,the fire department, the police department, an administrator of thebuilding that houses the elevator machine room, and/or an ambulancecorps, to name a few.

In embodiments, computing device 2702 may also verify sensor feedbackdata by correlating two seemingly unconnected sensor data in order togenerate and send notification messages. A process for verifying sensorfeedback data, in embodiments, is illustrated in connection with FIGS.45A and 45B. For example, referring to FIG. 45A, First IoT Device 4304-1and Second IoT Device 4304-2 may transmit sensor feedback data toComputing Device 2702 via network 100. Continuing the example, referringto FIG. 45B, the computing device 2702 may at step S4502 verify thesensor feedback data received from the First IoT Device 4304-1 andSecond IoT Device 4304-2 individually (e.g., steps S4502-1 verifyfeedback data from the First Computing Device and S4502-1 verifyfeedback data from the Second Computing Device 4502-2) and/or together.The sensor feedback data, for example, may include data that indicatesan error, such as, an error message, data with an excessive amount ofnoise, a lack of data, and/or a combination thereof, to name a few. Inembodiments, one or more processor(s) (e.g., processors of the computingdevice 2702, processor(s) of computer 1802, to name a few) may beoperable to verify sensor feedback data.

Following this paragraph are three examples of unverified sensorfeedback data (e.g., data which is at least partially defective). Thefirst example (Table 28), illustrates an embodiment where the sensorfeedback data values are zero, (e.g., indicating a defective sensor).Where the sensor feedback data values are zero, in embodiments, thecomputing device 2702 may determine that an IoT Device is at leastpartially defective and/or generate and send a message to anadministrator notifying of the IoT Device (e.g., to verify the IoTDevice's proper operation, to repair the IoT Device, and/or to replacethe IoT Device, to name a few). The second example (Table 29),illustrates an embodiment where the sensor feedback data is not receivedfrom an IoT Device (e.g., the IoT Device is not communicating data tothe computing device 2702). In embodiments, where sensor feedback datais not being received from an IoT Device, the computing device 2702 maygenerate and send a message to an administrator notifying of the IoTDevice (e.g., to verify the IoT Device's proper operation, to repair theIoT Device, and/or to replace the IoT Device, to name a few). The thirdexample (Table 30), illustrates an embodiment where the sensor feedbackdata is data values are bifurcative and/or oscillating between minimumand maximum and/or within a range (e.g., indicating no valid readoutvalues from the IoT Device). In embodiments, where sensor feedback datais not returning valid readout values, the computing device 2702 maygenerate and send a message to an administrator notifying of the IoTDevice (e.g., to verify the IoT Device's proper operation, to repair theIoT Device, and/or to replace the IoT Device, to name a few).

TABLE 28 First IoT Device 4304-1 Time (HH:MM) Sensor Feedback Data Value00:00 0 00:15 0 00:30 0 . . . . . . 23:45 0

TABLE 29 Second IoT Device 4304-2 Time (HH:MM) Sensor Feedback DataValue 00:00 n/a 00:15 n/a 00:30 n/a . . . . . . 23:45 n/a

TABLE 30 Third IoT Device 4304-3 Time (HH:MM) Sensor Feedback Data Value00:00  0 00:15 255 00:30  0 00:45 255 01:00  0 . . . . . . 23:45  0

In embodiments, the computing device 2702 may utilize one or morestandardized Euclidean Distances calculations (e.g., distance betweenvectors and/or arrays) to correlate to verify two seemingly unconnectedpieces of sensor feedback data. In embodiments, the computing device2702 may correlate two seemingly unconnected pieces of sensor feedbackdata utilize one or more standardized Euclidean Distances (SED)calculations by: (1) identifying two or more seemingly unconnectedpieces of sensor feedback data utilizing the results of the SEDcalculations; (2) generating a similarity metric for each piece ofsensor feedback data utilizing the results of the SED calculations; (3)generating a message including information regarding the correlatedsensor feedback data; and/or (4) a combination thereof, to name a few.In embodiments, alternatively to and/or in combination with the SEDcalculations, the computing device 2702 may utilize one or more of thefollowing: supervised learning algorithms (e.g. classificationsupervised learning, regression supervised learning), unsupervisedlearning algorithms (e.g. association unsupervised learning, clusteringunsupervised learning, dimensionality reduction unsupervised learning),reinforcement learning algorithms (e.g. through trial and error),semi-supervised algorithms, Naïve Bayes Classifier Algorithm(s), K MeansClustering Algorithm(s), Support Vector Machine Algorithm(s), AprioriAlgorithm(s), Linear Regression, Logistic Regression, Artificial NeuralNetworks, Random Forests, Decision Trees, Histogram of OrientedGradients Algorithm(s), and/or Nearest Neighbors, to name a few. Thefollowing table (Table 31) is an illustrative example where a portion ofthe sensor feedback data is determined to be a mistake and is discarded.

TABLE 31 Fourth IoT Device 4304-4 Time (HH:MM) Sensor Feedback DataValue 00:00 68 00:15 68 00:30 68 00:45 68 01:00 68 . . . 11:30 70 11:452,320 12:00 71 12:15 71 . . . 23:45 68

Referring to the above illustrative example, the sensor feedback datavalue received by the computing device 2702 from the fourth IoT Device4304-4 at 11:45 may be determined to be a mistake (e.g., the Fourth IoTDevice 4304-4 is not capable of reading a temperature of 2,320 degreesFahrenheit—exceeding a predetermined threshold; the temperature of aroom cannot feasibly fluctuate between 70 and 2,320 degrees Fahrenheitwithin 15 minutes, to name a few). In embodiments, the mistaken readingmay result in the computing device 2702 lowering the reliability ratingof the Fourth IoT Device 4304-4. In embodiments, the value at 11:45 maybe discarded by the Fourth IoT Device 4304-4 and/or redirected to afolder for readings from IoT Devices which were determined to bemistaken. In embodiments, at step S4505, the computing device 2702 maygenerate and send a message to an administrator notifying of the IoTDevice (e.g., to verify the IoT Device's proper operation, to repair theIoT Device, and/or to replace the IoT Device, to name a few).

In embodiments, the following exemplary code is run for each sensor toassist in verifying by determining a correlation between two or moreseemingly unconnected pieces of sensor feedback data in accordance withthe present invention:

Exemplary Pseudocode

#Define a function that fits model to training set (or prior known gooddata, evaluate performance with test set, and forecast with wholedataset def proph_it(train, test, w-hole, interval=0.95, forecastperiods 1, forecast_periods2):

“‘Uses Facebook Prophet to fit model to train set, evaluate performancewith test set, and forecast with whole dataset The model has a 95%confidence interval by default.

Remember: datasets need to have two columns, ‘ds’ and ‘y’, Dependencies:fbprophet, matplotlib.pyplot as pit

Parameters:

-   train: training data-   : testing/validation data-   whole: all available data for forecasting-   interval: confidence interval (percent)-   forecast_periods1: number of months for forecast on training data-   forecast_periods2: number of months for forecast on w hole dataset’”-   # Fit model to training data and forecast-   model=proph(interval_vvidth=interval)-   model,fit(train)-   future=model.make_future_dataframe(periods=forecast_periods1,    freq=′MS′)-   forecast=model, predict(future)-   # Plot the model and forecast-   model.plot(forecast, uncertainty=True)-   plt title(‘Training data with forecast’)-   plt.legend();-   # Make predictions and compare to test data-   y_pred=model.predict(test)-   # Plot the model, forecast, and actual (test) data-   model.plot(y_pred, uncertainty=True)-   plt.plot(test[‘ds’], test[‘y’], color=‘r’, label=‘actual’)-   plt.title(‘Validation data v. forecast’)-   pit legend();-   # Fit a new model to the whole dataset and forecast-   model2=proph(interval_width=0.95)-   model2.flt(whole)-   future2=model2.make_future_dataframe(periods=forecast periods2,    freq=‘MS’)-   forecast2=model2 predict(future2)-   # Plot the model and forecast-   model2.plot(forecast2, uncertainty=True)-   plt title(‘{}-month forecast’.format(forecast_periods2))-   plt legend();-   # Plot the model components-   model2.plot_components(forecast);-   return y_pred, forecast2

The above exemplary code was based on computer code for correlationbetween pieces of data available at:https://towardsdatascience.com/time-series-modeling-with-facebook-prophet-57f146a11d0d.

To validate and/or verify sensor feedback data, in embodiments, thecomputing device 2702 may executed the above pseudocode for each IoTdevice of the plurality of IoT devices 4302. For example, the computingdevice may at step S4504, determine the validity of sensor feedback datafrom the First IoT Device 4304-1 and/or the Second IoT Device 4304-2 (asillustrated in FIG. 45A) by executing the above pseudocode If, referringto FIG. 45B, for example, the sensor feedback data at S4504-1 andS4504-2 is determined to be not valid or not verified, the sensorfeedback data may be discarded. In embodiments, the data received anddiscarded may be corrupted and/or unreadable sensor feedback data.

If, in embodiments, the sensor feedback data is valid and/or verified,in embodiments, at step S4506, the valid and/or verified sensor feedbackdata is analyzed by the computing device 2702. In embodiments, thecomputing device may utilize results of the execution of the pseudocodeassociated with step S4504 in one or more SED calculations, e.g.,utilizing the following exemplary pseudocode:

Exemplary Pseudocode

-   Define period=3600 # for example, 60 minutes=3600 seconds-   Define Alarm_lndicator=0-   Define Sensor_Check_Array[]=0-   (sensor[], period) {-   If (Calc_ED(sensor[i].period.readings[],    sensor[i].period.forecast[]) >0.05)

Then

Alarm_Indicator +=1

-   Sensor_Check_Array[i++]=sensor[i].name-   End-   If Alarm_Indicator >3 # or any other desired threshold

Then

-   -   Send_message(receipient, “Multiple sensors (%s) are showing        unexpected readings concurrently. Please check”, Sensor_Check        Array)

In embodiments, referring to FIG. 45B, at step S4508, the computingdevice 2702 may determine that the sensor feedback data does notindicate an emergency situation. For example, the computing device S4508may analyze the sensor feedback data in the context of one or morepredetermined thresholds (e.g., above a threshold, below a threshold),one or more predetermined ranges (e.g., inside a predetermined range,outside a predetermined range), and/or a combination thereof (e.g.,above a threshold and within a range indicating a type of emergency isoccurring), to a name a few. For example, the computing device 2702 maydetermine the sensor feedback data (e.g., data from a temperaturesensor) of an IoT Device monitoring a refrigerator is below a specifiedthreshold (e.g., based on the contents of the refrigerator). As anotherexample, the computing device 2702 may determine sensor feedback data(e.g., data from a temperature sensor) of an IoT Device monitoring theambient temperature of a room is within a predetermined range (e.g., aspecified range around room temperature).

In embodiments, the computing device 2702 at step S4508 may determinethe sensor feedback data indicates an emergency. If, continuing theexample of a refrigerator, the IoT Device reads the temperature as abovethe specified threshold, the computing device 2702 may determine anemergency is occurring. In embodiments, the computing device 2702 mayfurther analyze the sensor feedback data to determine the type ofemergency (e.g., the refrigerator is on fire) and/or the urgency of theemergency (e.g., if a maintenance man gets to the refrigerator within 2hours, the contents of the refrigerator may be saved). If, continuingthe example of a room, the IoT Device reads the temperature as outsidethe predetermined range, the computing device may determine the room ison fire if the verified sensor data indicates an ambient temperatureabove a predetermined threshold.

In a possible emergency situation, in embodiments, at step S4510, thecomputing device 2702 may generate a message alerting one or more of anadministrator of the IoT Devices, emergency services, third parties,and/or a combination thereof, to name a few. This message, which may besimilar to the exemplary graphical user interfaces illustrated in FIGS.46A and 46B, at step S4512, may be sent to, for example, emergencyservices to alert said services of the possible emergency situation(e.g., as shown in FIG. 45A where the generated message is sent from thecomputing device 2702 to the emergency service(s) 4502). In embodiments,the message and/or notification recipient may have predeterminedresponse options—e.g., acknowledge, dismiss, request more details,monitor (e.g., request frequent updates and/or live streaming of thedata from the IoT Device), and/or a combination thereof, to name a few.

In embodiments, computing device 2702 may associate and/or correlatedata received from two or more IoT Devices of the plurality of IoTDevices 4302. The computing device 2702 may, for example, utilize aplurality of IoT Devices 4302 to determine a particular person(particular animal, and/or particular object) is in a room. Continuingthe example, a temperature sensor 4328 that reads within a 2m radius ofits location detects an increase in temperature. The temperature sensor4328, continuing the example, is co-located with a Bluetooth® receiverand a computer vision camera. The computing device 2702, finishing theexample, may determine that a particular person is present if one ormore of the following are true at or around (e.g., within a range) thesame time: (1) the Bluetooth® receiver indicates a cell phone associatedwith the particular person is within 2 meters and/or (2) the particularperson is identified by the computer vision camera (e.g., utilizingfacial recognition software). The detection of that particular personmay validate the increased temperature readings from the temperaturesensor 4328. In embodiments, the computing device 2702 may erroneouslyread that a person (and/or more than one person) is present. Inembodiments, the mistaken reading may be utilized to determine the IoTdevice(s) associated with the mistaken feedback data are at leastpartially defective.

In embodiments, the computing device 2702 may use the feedback data fromthe plurality of IoT Devices 4302 to recognize an object and/or aparticular identifier (e.g., barcode, NFC, RFID, MAC address, and/or acombination thereof, to name a few) associated with an object todetermine the validity of sensor feedback data. In embodiments, one ormore of the plurality of IoT Devices 4302 may be able to move about oneor more axis, focus on one or more points, and/or utilize a mask (e.g.,an image mask) such that only a portion of the area covered by theparticular IoT device is covered. In such embodiments, for example, thecomputing device 2702 may correlate sensor feedback data from two ormore IoT devices and/or generate masking instructions to send to one ormore IoT devices to validate sensor feedback data.

As another example, a current sensor may indicate a brief high currentevent on floor 5 of a building, and a thermometer may record a slightincrease in temperature on floor 6 twenty minutes later. Each of theseevents, analyzed individually, may not be significant, however incombination they may indicate that a short circuit on floor 5 sparked afire on floor 5 which is what is causing floor 6 to start warming up.Continuing the example, computing device 2702 may send a notification toone or more of the following: an administrator of the computing device2702, the fire department, the police department, an administrator ofthe building that houses the sensor, room, and/or an ambulance corps, toname a few.

The process for determining whether at least one sensor device of thefirst group of sensor devices is at least partially defective maycontinue with step S4432. At step S4432, the computing device 2702and/or sensor module 4306 may compare the first respective sensorfeedback information with second respective sensor feedback informationassociated with one or more other sensor devices of the first group ofsensor devices. Continuing the example, the sensor module 4306 and/orcomputing device 2702 may extract stored sensor information ofThermometer A and/or Thermometer B and data from the obtained sensorfeedback data of Thermometer A and/or Thermometer B. For the purposes ofthis example, the exemplary information extracted by the computingdevice 2702 is listed in the below table:

TABLE 32 Feedback Data of Feedback Data of Feedback Data of Thermometer4344 Thermometer A Thermometer B Temperature Time Temperature TimeTemperature Time 134° F. 1:00 PM   68° F. 1:00 PM 81.2° F. 1:00 PM  76°F. 2:00 PM 67.4° F. 2:00 PM 80.4° F. 2:00 PM 121° F. 3:00 PM 72.6° F.3:00 PM 81.3° F. 3:00 PM 143° F. 4:00 PM 71.1° F. 4:00 PM 80.8° F. 4:00PM

In embodiments, the computing device 2702 (and/or sensor module 4301)may determine that the feedback data of Thermometer A and Thermometer Bare within the 60° F. through 90° F. expected range. This determination,in embodiments, may allow computing device 2702 to determine there is noemergency and that Thermometer 4344 is at least partially defective.

In embodiments, the feedback data may indicate that thermometer 4344 isproducing data within an expected range. However, the computing device2702 may determine the thermometer 4344 is at least partially defectivebased on one or more of the following: (1) a timestamp, (2) connectioninformation, (3) a current reliability rating, (4) useful lifeinformation, (5) sensor variance information, (6) sensor deviationinformation, (7) historical reliability rating and/or (8) deviceinformation, to name a few. In embodiments, IoT devices may experienceone or more of the following defects: hardware defects (e.g. batterylife, wiring, connectivity issues, broken parts, wear and tear, etc.),software defects, and/or malicious defects (e.g. a hacker or other badactor, etc.) to name a few.

In embodiments, as mentioned above, the process for determining whetherat least one sensor device is at least partially defective may continuewith obtaining respective historical reliability ratings for each sensordevice of the first group of sensor devices. Continuing the example, thecomputing device 2702 and/or the sensor module 4306, may extracthistorical reliability from the stored sensor information associatedwith thermometer 4344, Thermometer A, and Thermometer B. In embodiments,the historical reliability ratings may indicate that the first group ofIoT devices 4304 were all assigned a reliable reliability rating

In embodiments, the respective historical reliability ratings may becompared to the determined and assigned reliability ratings. Continuingthe example, the historical reliability ratings of reliable, incombination with the previous analysis, may indicate that thethermometer 4344 is at least partially defective. In embodiments, aprevious reliable reliability rating may indicate that the sensor isdetecting an emergency—which may result in the same actions noted above.

In embodiments, the computing device 2702, utilize hierarchicalinformation associated with the plurality of IoT devices 4302 to predictif a defect and/or emergency will spread from the at least one IoTDevice to one or more of the remaining IoT devices of the plurality ofIoT Devices 4302. Continuing the example above with the eleven IoTDevices monitoring a five story apartment building, the respectivesensor types may be as follows:

TABLE 33 IoT Device Sensor Type First IoT Device Electrical CurrentSensor Second IoT Device Electrical Current Sensor Third IoT DeviceElectrical Current Sensor Fourth IoT Device Electrical Current SensorFifth IoT Device Electrical Current Sensor Sixth IoT Device ElectricalCurrent Sensor Seventh IoT Device Temperature Sensor Eighth IoT DeviceTemperature Sensor Ninth IoT Device Temperature Sensor Tenth IoT DeviceTemperature Sensor Eleventh IoT Device Temperature Sensor

Continuing the example, the 2^(nd) IoT Device may transmit sensorfeedback data that indicates a loss of power. A loss of power to anentire floor of an apartment building, in embodiments, may causeinconsistent feedback data associated with the remaining IoT Devices(1^(st), 3^(rd)-11^(th)). For example, the second floor of the apartmentbuilding may lose power in the middle of a hot summer day. Withoutpower, in embodiments, the second floor may also lose air conditioning,which may cause the 8^(th) IoT device to gather and transmit feedbackdata that indicates an increase in temperature over a period of time.Additionally, in embodiments, an increase of heat during a hot summerday may also cause the 3^(rd), 4^(th), and 5^(th) floors to increase intemperature and/or electrical output. For example, as the higher floorsincrease in temperature, tenants of the apartments may turn up theirrespective air conditioners, resulting in an increased amount ofelectrical current. In embodiments, because the power outage affectedthe 2^(nd) floor, the 1^(st) floor may not experience an increase intemperature and/or an increase in electrical output. The computingdevice 2702, in embodiments, may analyze the change in temperature andelectrical output over time to determine whether the 2nd IoT Deviceand/or 8^(th) IoT device are defective or whether the 2^(nd) IoT Deviceand/or 8^(th) IoT device are indicating an emergency situation. Forexample, if the temperature increases below a threshold rate (e.g. agradual rate), the temperature increase may indicate that the 2^(nd) IoTDevice and/or the 8^(th) IoT device is defective. Alternatively, forexample, if the temperature increases above a threshold rate (e.g. afast rate), the temperature increase may indicate an emergency situationon the 2^(nd) floor of the apartment building. The computing device2702, in embodiments, may also predict the effects a defective IoTdevice on the 2^(nd) floor will have on the remaining IoT Devices. Inembodiments, the predicted effects may be stored and accessed whenfeedback data is received from the plurality of IoT devices.

Referring to FIG. 44B, in embodiments the process for determiningwhether at least one sensor device is at least partially defective maycontinue with step S4426. At step S4426, the computing device 2702 maygenerate a notice message. The notice message, in embodiments, mayidentify the at least one sensor device that is at least partiallydefective. In embodiments, the computing device 2702 may also generate areport regarding the at least one partially defective IoT device. Thereport, in embodiments, may include one or more of the following: (1)sensor information associated with the at least one partially defectiveIoT device, (2) a repair message indicating the at least one partiallydefective IoT device, and/or (3) the sensor feedback informationassociated with the at least one partially defective IoT device, to namea few. The repair message may indicate one or more of the following: (1)the issue associated with the IoT device, (2) whether a purchase orderhas been sent for necessary parts and/or replacement (see stepsS4464-S4468, the descriptions of which applying herein), and/or (3) anacceptable timeframe to repair the at least one partially defective IoTdevice, to name a few.

In embodiments the process for determining whether at least one sensordevice is at least partially defective may continue with step S4426. Atstep S4426, the computing device 2702 transmits the notice message to adevice associated with at least one administrator of the computingdevice 2702 via network 100. In embodiments, the computing device 2702may transmit the aforementioned generated report to the deviceassociated with at least one administrator of the computing device 2702via network 100.

Alternatively, in embodiments, the computing device may determine thatno IoT device is at least partially defective. In embodiments, theprocess may continue from step S4424 with step S4426′. At step S4426′,the computing device 2702 may generate first machine-readableinstructions to provide a first graphical user interface. The firstgraphical user interface, in embodiments, may include a display of oneor more of the following: (1) the sensor feedback information associatedwith the first group of IoT devices 4304, (2) the sensor feedbackinformation associated with the plurality of IoT devices 4302, (3) thesensor information associated with the first group of IoT devices 4304,and/or (4) the sensor information associated with the plurality of IoTdevices 4302, to name a few. The first machine-readable instructions, inembodiments, may include instructions that the cause the first graphicaluser interface to be displayed upon receipt by an electronic device witha display screen.

In embodiments, the process may continue from with step S4428′. At stepS4428′, the computing device 2702 may transmit the firstmachine-readable instructions to a device associated with at least oneadministrator of the computing device 2702. Upon receipt, the device maydisplay the first graphical user interface.

In embodiments, the process for determining whether at least one sensordevice is at least partially defective may optionally continue with stepS4434 of FIG. 44D. Referring to FIG. 44D, step S4434, the computingdevice 2702 may identify respective feedback data associated with one ormore sensors that are rated as reliable. Continuing the example withthree thermometers, the computing device 2702 may identify Thermometer Aand Thermometer B as reliable IoT devices of the first group of IoTdevices 4304.

The process may continue with step S4436. In embodiments, at step S4436,the computing device 2702 may generate first machine-readableinstructions to provide a first graphical user interface. The firstgraphical user interface, in embodiments, may include a display of oneor more of the following: (1) the sensor feedback information associatedwith the reliable IoT devices (e.g. Thermometer A and Thermometer B),(2) the feedback data associated with the reliable IoT devices (e.g.Thermometer A and Thermometer B), and/or (3) the sensor informationassociated with the reliable IoT devices (e.g. Thermometer A andThermometer B), to name a few. The first machine-readable instructions,in embodiments, may include instructions that the cause the firstgraphical user interface to be displayed upon receipt by an electronicdevice with a display screen. In embodiments, the first graphical userinterface may also include feedback data from IoT devices that aremarked as unreliable. In embodiments, where unreliable feedback data isdisplayed, the computing device 2702 may highlight the unreliable and/orreliable feedback data and/or the computing device 2702 may weigh thereliable feedback data more in its presentation of the feedback data.

In embodiments, the process may continue from with step S4438. At stepS4438, the computing device 2702 may execute the first machine-readableinstructions to present the first graphical user interface on a displayscreen associated with the computing device. The display screendescribed herein may be similar to the monitor 1823 described above inconnection with FIG. 18 , the description of which applying herein.

In embodiments, the computing device may transmit the firstmachine-readable instructions to one or more of the following: a deviceassociated with at least one administrator of the computing device 2702,a device associated with the owner of the IoT devices, and/or a deviceassociated with a third party, to name a few. Upon receipt, the devicemay display the first graphical user interface.

In embodiments, the process for determining whether at least one sensordevice is at least partially defective may optionally continue with oneor more of the processes described in connection with FIGS. 45E, 45F,and 45G. Each process may be executed contemporaneously and/orseparately. In embodiments, each process may be optionally executedafter step S4425.

In embodiments, the computing device 2702 and/or the sensor module 4306may flag one or more IoT devices that are at least partially defectiveand/or unreliable. Referring to FIG. 44E, in embodiments the process forflagging at least one sensor device that is at least partially defectiveand/or unreliable may optionally begin with step S4440. At step S4440,in embodiments, the sensor module 4306 and/or the computing device 2702may obtain respective sensor identification information associated withthe at least one sensor device that is at least partially defective.Continuing the example, the computing device 2702 may obtain the sensorinformation associated with the thermometer 4344. Once obtained, thecomputing device 2702 may extract the sensor identification informationassociated with thermometer 4344. As shown above and reproduced below,for example, the thermometer 4344 may have the following sensoridentification information:

-   Thermometer 4344 Sensor Identification Information-   Unique Name - First Floor Thermometer-   Sensor Type - Thermometer

In embodiments, the computing device 2702 may also create a second groupof IoT devices. The second group of IoT devices may be each IoT deviceof the first group of IoT devices 4304 that is not at least partiallydefective. Continuing the example, the second group of IoT devices maybe Thermometer A and Thermometer B.

In embodiments the process for flagging at least one sensor device thatis at least partially defective and/or unreliable may optionallycontinue with step S4442. At step S4442, in embodiments, the sensormodule 4306 and/or the computing device 2702 may store the extractedsensor identification information in a flagged sensor databaseoperatively connected to the computing device 2702. In embodiments, thesensor identification information may be stored with other IoT devicesthat have been or will be flagged as at least partially defective and/orunreliable, in one or more of the following: a flagged sensor database,memory 2702-3, memory of the sensor module 4306, and/or the memorydevice 4308, to name a few.

In embodiments the process for flagging at least one sensor device thatis at least partially defective and/or unreliable may optionallycontinue with step S4444. At step S4444, in embodiments, the sensormodule 4306 and/or the computing device 2702 may generate secondmachine-readable instructions. In embodiments, the second machinereadable instructions may be instructions to redirect all receivedsensor feedback information that is associated with the at least onesensor device that is at least partially defective and/or unreliable.The instructions to redirect the sensor feedback information, inembodiments, may cause all received sensor feedback information that isassociated with the at least one sensor device that is at leastpartially defective and/or unreliable to be stored with the respectivesensor identification information (e.g. in the flagged sensor database).Continuing the example, until the thermometer 4344 is repaired and/orreplaced, the redirect instructions may cause all sensor feedbackinformation received from thermometer 4344 by the computing device 2702to be transmitted to the flagged sensor database (e.g. skipping stepsS4414-S4428 in the next iteration of the currently describedprocess(es)).

In embodiments, the following is exemplary code for redirecting flaggedsensor feedback information in accordance with exemplary embodiments ofthe present invention:

Exemplary Pseudocode

-   import numpy as np-   import pandas as pd-   from pandas import Series-   from pandas import DataFrame-   from skleam.metrics.pairwise import euclidean_distances-   from skleam.metrics.pairwise import paired_distances-   from scipy.spatial.distance import cdist-   d={‘coll’: [0, 1,2, 3, 4, 5,6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,    17, 18, 19, 20,21,22],-   ‘col2’: [0, 1,1, 1,2, 1, 1,2, 3, 2, 1, 1, 1, 1, 1,2, 1, 1,3,4,    1,2,3]}-   df=pd.DataFrame(data=d)-   d2={‘coll’: [0, 1,2], ‘col2': [1,2, 3[}-   patch=pd.DataFrame(data=d2)-   def orig(df, patch):

df[‘corr’]=np.nan

for i in range(df.shape[0]):

-   -   #select the df window with the same size of patch    -   window=df[i:i+patch.shape[0]]    -   #If window and patch have different shapes —>Break    -   if window.shape[0]!=patch.shape[0]:        -   break    -   else:        -   patch reset_index(inplace=True, drop=True)        -   window.reset_index(inplace=True, drop=True)        -   df[‘corr’]=cdist(df[[‘col2’]], patch[[‘col2’]],‘euclidean’)    -   if df[‘corr’]<THRESHOLD :        -    move(df, bad_readings_folder)

The above exemplary code was based on computer code for pattern findingwithin a time series graph available at:https://stackoverflow.com/questions/55057635/find-pattern-in-time-series-graph-with-pandas.

In embodiments, the computing device 2702 may generate a plurality ofmachine-readable instructions, each of the plurality being for each IoTdevice of the second group of IoT devices and/or plurality of IoTdevices 4302 less the at least one partially defective and/or unreliableIoT device. The machine-readable instructions, in embodiments, mayinclude instructions to ignore all transmissions from the at least oneIoT device that is at least partially defective and/or unreliable. Inembodiments, the instructions may include a direction to ceasetransmitting data to the at least one IoT device that is at leastpartially defective and/or unreliable. In embodiments, the plurality ofmachine-readable instructions may include the extracted sensoridentification information associated with the at least one IoT devicethat is at least partially defective and/or unreliable.

Once generated, in embodiments at optional step S4446, the plurality ofmachine-readable instructions may be transmitted by the computing device2702 to each IoT device of the second group of IoT devices 4304 vianetwork 100. In embodiments, upon receipt of respective machine-readableinstructions, the respective IoT device may execute the instructions andstore the extracted sensor identification information associated withthe at least one IoT device that is at least partially defective and/orunreliable rating to ignore all transmissions from said extracted sensoridentification information and cease transmitting to the at least oneIoT device.

In embodiments the process for flagging at least one sensor device thatis at least partially defective and/or unreliable may optionallycontinue with step S4448. At step S4448, in embodiments, the sensormodule 4306 and/or the computing device 2702 may execute thesecond-machine readable instructions, flagging the at least one IoTdevice and redirecting the respective sensor feedback information.

In embodiments, the computing device 2702 may determine that the atleast one IoT device that is at least partially defective and/orunreliable can be fixed using troubleshoot instructions. In embodiments,the troubleshoot instructions may be stored with the respective sensorinformation. Referring to FIG. 44F, in embodiments the process fortroubleshooting an IoT device may optionally begin with step S4450. Atstep S4450, in embodiments, the sensor module 4306 and/or the computingdevice 2702 may generate second machine-readable instructions. Inembodiments, the second machine readable instructions may includetroubleshoot instructions associated with the at least one sensor devicethat is at least partially defective. The troubleshoot instructions, inembodiments, may be extracted from sensor information associated withthe at least one IoT device. In embodiments, the troubleshootinstructions may be obtained via a third party (e.g. third partydatabase 4310).

In embodiments the process for troubleshooting an IoT device mayoptionally continue with step S4452. At step S4452, in embodiments, thesensor module 4306 and/or the computing device 2702 may transmit thesecond machine-readable instructions to the at least one IoT device thatis at least partially defective and/or unreliable via network 100. Uponreceipt of the second machine-readable instructions, in embodiments, theat least one IoT device may execute the second machine-readableinstructions, causing the at least one IoT device to troubleshoot itssoftware.

In embodiments, at step S4454, the computing device 2702 may obtainsecond sensor feedback information from the at least one IoT device thatis at least partially defective and/or unreliable. Step S4454, inembodiments, may be similar to step S4410 described above in connectionwith FIG. 44A, the description of which applying herein. In embodiments,the second sensor feedback information may be similar to the sensorfeedback information described above in connection with FIGS. 44A-44D,the description of which applying herein.

In embodiments, the process for troubleshooting an IoT device maycontinue with step S4456. At step S4456, the sensor module 4306 and/orthe computing device 2702 may generate a third timestamp. The thirdtimestamp, in embodiments, may indicate a time at which the secondsensor feedback information was obtained. In embodiments the sensorfeedback information and third timestamp are stored by the computingdevice 2702 and/or the sensor module 4306. In embodiments, the secondsensor feedback information and/or the third timestamp may be stored,separately and/or together, in one or more of the following: the flaggedsensor database, memory 2702-3, memory of the sensor module 4306, and/orthe memory device 4308, to name a few. In embodiments, the computingdevice 2702 may also obtain second additional information from one ormore third party database(s) 4310. The second additional information andmanner of obtaining said information may be similar to the descriptionof additional information (e.g. weather information) described above inconnection with FIGS. 44A-44D, the description of which applying herein.

In embodiments, the process for troubleshooting an IoT device continuewith step S4458. At step S4458, the sensor module 4306 and/or thecomputing device 2702 may generate second sensor variance informationfor the at least one IoT device. The second sensor variance informationmay be based on at least one of the following: the second sensorfeedback information, the third timestamp, the second additionalinformation, the sensor feedback information, the sensor information,the aforementioned additional information (e.g. weather information),the first timestamp (e.g. associated with gathering and/or transmittingof the sensor feedback information), and/or the second timestamp (e.g.associated with the receipt of the sensor feedback information, to namea few. The generation of the second sensor variance information for theat least one IoT device may be similar to the generation of the sensorvariance information described above in connection with step S4416 ofFIG. 44A, the description of which applying herein.

In embodiments, the process for troubleshooting an IoT device continuewith step S4460. At step S4460, the sensor module 4306 and/or thecomputing device 2702 may generate second sensor deviation informationfor the at least one IoT device. In embodiments, the second sensordeviation information may be based on at least one of the following: thesecond sensor feedback information, the second sensor varianceinformation, the third timestamp, the sensor feedback information, thesensor information, the aforementioned additional information (e.g.weather information), the first timestamp (e.g. associated withgathering and/or transmitting of the sensor feedback information),and/or the second timestamp (e.g. associated with the receipt of thesensor feedback information, to name a few. The generation of the secondsensor deviation information for the at least one IoT device may besimilar to the generation of the sensor deviation information describedabove in connection with step S4418 of FIG. 44A, the description ofwhich applying herein.

In embodiments, the process for troubleshooting an IoT device continuewith step S4462. At step S4462, the sensor module 4306 and/or thecomputing device 2702 may determine whether the at least one IoT deviceremains at least partially defective. Determining whether the at leastone IoT device remains partially defective may be similar to step S4424described above in connection with FIGS. 44B and 44C, the description ofwhich applying herein.

In embodiments, the sensor module 4306 and/or the computing device 2702may determine a current reliability rating of the at least one IoTdevice. In embodiments, the determination of the current reliabilityrating for the at least one IoT device may be similar to step S4422described above in connection with FIGS. 44A-44D, the description ofwhich applying herein.

In embodiments, once the at least one IoT device has received thetroubleshoot instructions and the computing device 2702 and/or thesensor module 4306 have determined whether the at least one IoT deviceis defective, the computing device 2702 may generate and send an updatednotice. The updated notice, in embodiments, may identify whether the atleast one sensor device is still at least partially defective. Inembodiments, the computing device 2702 may also generate and send anupdated report regarding the at least one partially defective IoTdevice. The report, in embodiments, may include one or more of thefollowing: (1) sensor information associated with the at least onepartially defective IoT device, (2) a repair message indicating the atleast one partially defective IoT device, (3) second sensor feedbackinformation, and/or (4) the sensor feedback information associated withthe at least one partially defective IoT device, to name a few. Therepair message may indicate one or more of the following: (1) the issueassociated with the IoT device, (2) whether a purchase order has beensent for necessary parts and/or replacement (see steps S4464-S4468, thedescriptions of which applying herein), and/or (3) an acceptabletimeframe to repair the at least one partially defective IoT device, toname a few. In embodiments, the updated notice and/or updated report maybe transmitted from the computing device 2702 to the device associatedwith the at least one administrator.

In embodiments, if the at least one IoT device remains at leastpartially defective and/or unreliable, the computing device 2702 mayrepeat this process with second troubleshoot instructions. Inembodiments, if the at least one IoT device is no longer at leastpartially defective and/or unreliable, the process may continue withstep S4426′ of FIG. 45B.

In embodiments, the computing device 2702 and/or sensor module 4306 maydetermine one or more parts and/or a replacement IoT device are requiredfor repairs to the at least one IoT device. Referring to FIG. 44G, inembodiments the process for purchasing parts and/or a replacement IoTdevice may optionally begin with step S4464. At step S4464, inembodiments, the sensor module 4306 and/or the computing device 2702 mayobtain respective sensor information associated with the at least onesensor device that is at least partially defective and/or unreliable.Once obtained, the computing device 2702 and/or sensor module 4306 mayextract a purchase order associated with the part and/or IoT device.

Continuing with step S4466, the computing device 2702 may generate apurchase order associated with repairs to the at least one sensordevice. For example, the computing device 2702 may use information fromthe respective sensor information to populate the purchase order. Inembodiments, the information may include the information illustrated inconnection with FIG. 43E. In embodiments, the computing device may haveaccess to payment methods for purchase orders (e.g. a credit cardnumber, bank account, etc.) and use said payment methods with thepurchase order. In embodiments, the computing device 2702 may have astored value, of which the purchase order may not exceed. The storedvalue, in embodiments, may prevent the computing device 2702 fromspending too much capital to repair and/or replace the at least one IoTdevice. Continuing with step S4468, the computing device 2702 may sendthe generated purchase order to a third party vendor via network 100. Inembodiments, the computing device 2702 may extract the contactinformation from the information illustrated in FIG. 43E.

Continuing with step S4470, the computing device 2702 may generate areport. In embodiments, the report may include one or more of thefollowing: a copy of the purchase order, the respective sensorinformation, and/or the repair message, to name a few. At step S4472,the computing device 2702 may send the report to the device associatedwith the at least one administrator and/or a device associated with theperson or persons who will be responsible for repairing and/or replacingthe IoT device.

In embodiments, the steps of the processes described above in connectionwith FIGS. 44A-44G may be rearranged and/or omitted.

In embodiments, a process for detecting at least a partially defectivesensor device may include: (a) providing a plurality of sensor devicesoperatively connected via a network to a computing device; (b) providinga reliability rating database operatively connected to the computingdevice, wherein the reliability rating database comprises a respectivehistorical reliability rating for each sensor device of the plurality ofsensor devices; (c) storing, by the computing device, in at least onememory element operatively connected to the computer device, sensorinformation associated with each sensor device of the plurality ofsensor devices, wherein the sensor information includes, for each sensordevice of the plurality of sensor devices: (1) sensor identificationinformation unique to each sensor device; (2) location informationindicating sensor location; (3) data range information indicating arange of sensor values, the data range information including: (i) amaximum sensor value; and (ii) a minimum sensor value; (d) accessing, bya sensor module of the computing device, the sensor information; (e)selecting, by the sensor module, at least a first group of sensordevices of the plurality of sensor devices based on at least the sensorinformation, (f) obtaining, by the sensor module, first sensor feedbackinformation from one or more sensor devices of the first group of sensordevices, wherein the sensor feedback information includes: (1)identification information associated with a respective sensor deviceproviding the first sensor feedback information; (2) first feedback dataprovided by the respective sensor device in response to stimuli; and (3)a first timestamp indicating a first time at which the first feedbackdata was generated by the respective sensor device; (g) generating, bythe sensor module, a second timestamp indicating a time at which thefirst sensor feedback data was obtained by the sensor module; (h)storing, by the sensor module, the first sensor feedback information,including the second timestamp in the at least one memory element; (i)generating, by the sensor module, for each sensor device of the firstgroup of sensor devices, sensor variance information based on at leastthe first sensor feedback information; (j) generating, by the sensormodule for each sensor device of the first group of sensor devices,sensor deviation information based on at least the first sensor feedbackinformation and the generated sensor variance information; (k) storing,by the sensor module in the at least one memory element, respectivesensor variance information and respective sensor deviation informationassociated with each sensor device of the first group of sensor devices;(l) determining, by the computing device, a determined reliabilityrating of each sensor device of the first group of sensor devices basedon at least the following: (1) respective sensor variance information;(2) respective sensor deviation information; and (3) respective firstsensor feedback information; (m) determining, by the computing device,whether at least one sensor device of the first group of sensor devicesis at least partially defective based on at least the following: (1)comparing, by the computing device, first respective sensor feedbackinformation associated with the at least one sensor device with firstrespective data range information associated with the at least one othersensor device; (2) comparing, by the computing device, the firstrespective sensor feedback information with second respective sensorfeedback information associated with one or more other sensor devices ofthe first group of sensor devices; and (3) analyzing respectivedetermined reliability rating, wherein the computing device determinesthat the at least one sensor device is at least partially defective whenat least one of the following occurs: (A) the computing devicedetermines that the first respective sensor feedback data varies by morethan a first predetermined threshold from the first respective datarange information; (B) the computing device determines that the firstrespective sensor feedback data is inconsistent with the secondrespective sensor feedback data; and (C) the computing device determinesthat the first respective reliability rating indicates the at least onesensor device is unreliable; (n) generating, by the computing device, anotice message identifying the at least one sensor device that is atleast partially defective, where the notice message includes causeinformation indicating the at least one sensor device is at leastpartially defective; and (o) sending the notice message to a deviceassociated with at least one user of the computing device.

In embodiments, the computing device weighs respective sensor feedbackinformation associated with one or more sensor devices of the pluralityof sensor devices that are associated with a first respectivereliability rating indicating that one or more sensor devices arereliable more than respective sensor feedback information associatedwith one or more sensor devices of the plurality of sensor devices thatare associated with a second respective reliability rating indicatingthat one or more sensor devices are unreliable.

In embodiments, determining whether the at least one sensor device ispartially defective may further comprise: (4) obtaining, by thecomputing device from the reliability rating database, respectivehistorical reliability ratings for each sensor device of the first groupof sensor devices. In embodiments, in the event that both the obtainedreliability rating and the determined reliability rating indicate therespective sensor device is unreliable, the respective sensor device isat least partially defective. In embodiments, in the event that both theobtained reliability rating indicates the respective sensor device isreliable and the determined reliability rating indicates the respectivesensor device is unreliable, the respective sensor device is not atleast partially defective. In embodiments, in the event that both theobtained reliability rating indicates the respective sensor device isunreliable and the determined reliability rating indicates therespective sensor device is reliable, the respective sensor device isnot at least partially defective.

In embodiments, the method further comprises: (p) identifying, by thecomputing device, respective feedback data associated with one or moresensors that are rated as reliable; (q) generating, by the computingdevice, first machine-readable instructions to provide a first graphicaluser interface, wherein the first graphical user interface presents adisplay the respective feedback data associated with the one or moresensors that are rated as reliable; and (r) presenting, by the computingdevice based on the first machine-readable instructions, the firstgraphical user interface on a display screen associated with thecomputing device.

In embodiments, the method further comprises: (p) generating, by thecomputing device, second machine-readable instructions includingtroubleshoot instructions, wherein the troubleshoot instructions areassociated with the at least one sensor device; (q) sending, by thesensor module to the at least one sensor device via the network, thesecond machine-readable instructions, wherein, upon receipt of thesecond machine-readable instructions, the at least one sensor deviceperforms a troubleshoot operation by executing the secondmachine-readable instructions; (r) obtaining, by the sensor module fromthe at least one sensor device, second sensor feedback information; (s)generating, by the sensor module, a third timestamp indicating a time atwhich the second sensor feedback information was obtained by the sensormodule; (t) generating, by the sensor module, for the at least onesensor device, updated sensor variance information based on at least thesecond sensor feedback information and the sensor feedback information;(u) generating, by the sensor module for the at least one sensor device,updated sensor deviation information based on at least the sensorfeedback information, the generated sensor variance information, thesecond sensor feedback information, the updated sensor varianceinformation and the sensor deviation information; (v) determining, bythe computing device, whether at least one sensor device of the firstgroup of sensor devices is at least partially defective based on atleast the following: (1) comparing, by the computing device, firstrespective second sensor feedback information associated with the atleast one sensor device with the first respective data range informationassociated with the at least one other sensor device; (2) comparing, bythe computing device, the first respective second sensor feedbackinformation with second respective second sensor feedback informationassociated with one or more other sensor devices of the first group ofsensor devices; and (3) analyzing at least the updated sensor varianceinformation and the updated sensor deviation information, wherein thecomputing device determines that the at least one sensor device is atleast partially defective when at least one of the following occurs: (A)the computing device determines that the first respective second sensorfeedback information varies by more than the first predeterminedthreshold from the first respective data range information; (B) thecomputing device determines that first respective second sensor feedbackinformation is inconsistent with the second respective second sensorsecond feedback information; and (C) the computing device determinesthat the sensor variance information and the updated sensor deviationinformation indicate the at least one sensor device is unreliable; (w)generating, by the computing device, an updated notice messageidentifying whether the at least one sensor device is at least partiallydefective; and (x) sending the updated notice message to at least oneadministrator of the computing device.

In embodiments, the method further comprises: (p) obtaining, by thecomputing device, respective sensor information associated with the atleast one sensor device that is at least partially defective; (q)generating, by the computing device, a report comprising at least thefollowing: (1) the respective sensor information; and (2) a repairmessage indicating at least that the at least one sensor device is atleast partially defective; and (r) sending, by the computing device tothe device associated with the at least one user, the report. Inembodiments, the method further comprises: (s) sending, by the computingdevice to an electronic device operatively connected to the computingdevice, the report.

In embodiments, the method further comprises: (p) obtaining, by thecomputing device, respective sensor information associated with the atleast one sensor device that is at least partially defective; (q)determining, by the computing device based on the sensor information, atleast one item to purchase, wherein the one item is associated with theat least one sensor device; (r) generating, by the computing device, apurchase order comprising at least the following: (1) the respectivesensor information associated with the at least one sensor device thatis at least partially defective; and (2) the at least one item; and (s)sending, by the computing device, the purchase order to a third partyvendor. In embodiments, the method further comprises: (t) generating, bythe computing device, a report comprising at least the following: (1)the respective sensor information; (2) information identifying the atleast one item; and (3) a repair message indicating at least thefollowing: (i) that the at least one sensor device is at least partiallydefective; (ii) the at least one item; (u) sending, by the computingdevice to the device associated with the at least one user, the report.In embodiments, the method further comprises: (v) sending, by thecomputing device to an electronic device operatively connected to thecomputing device, the repair message.

In embodiments, the method further comprises: (p) obtaining, by thecomputing device, respective sensor identification informationassociated with the at least one sensor device that is at leastpartially defective; (q) storing, in a flagged sensor database of thecomputing device, the respective sensor identification information; (r)generating, by the computing device, second machine-readableinstructions to: (1) flag, by the flagged sensor database, obtainedrespective sensor feedback data associated with the sensoridentification information stored in the flagged sensor database; and(2) store, in the flagged sensor database, the flagged respective sensorfeedback data; and (s) executing, by the computing device, the secondmachine-readable instructions. In embodiments, the flagged respectivesensor feedback data is stored with the respective sensor identificationinformation. In embodiments, the method further comprises: (t)generating, by the computing device, third machine-readable instructionsto: (1) ignore data received from the at least one sensor device that isat least partially defective; and (2) cease transmitting data to the atleast one sensor device that is at least partially defective; and (u)sending, by the computing device to each sensor device of a second groupof sensor devices, the third machine-readable instructions, wherein thesecond group of sensor devices is the first group of sensor devices lessthe at least one sensor device, and wherein, upon receipt of the thirdmachine-readable instructions, each of the second group of sensordevices executes the third machine-readable instructions.

In embodiments, the method further comprises: (p) generating, by thecomputing device, a reliability report comprising, for each sensordevice of the first group of sensor devices, at least: (1) a respectivereliability rating; and (2) respective feedback data; and (q)transmitting, by the computing device to the device associated with theat least one user, the reliability report.

In embodiments, the method further comprises: (p) obtaining, by thecomputing device for each sensor device of the first group of sensordevices, first weather information associated with at least a firstrespective temperature at the first time within a predetermined radiusof respective location information, wherein the sensor varianceinformation is further based on the first weather information, andwherein the sensor deviation information is further based on the firstweather information.

In embodiments, the first group of sensor devices is selected based onlocation information.

In embodiments, the first group of sensor devices is selected based onthe sensor identification information.

In embodiments, the location information, for each sensor device of theplurality of sensor devices, includes respective hierarchicalinformation associated with a respective sensor device. In embodiments,the first group of sensor devices is selected based at least in part onrespective hierarchical information.

In embodiments, the first group of sensor devices selected by the sensormodule by performing the following steps: (1) determining, by the sensormodule for each sensor device of the plurality of sensor devices,respective location information; (2) selecting, by the sensor module, agroup of sensor devices of the plurality of sensor devices based on therespective location information, wherein each sensor device of the groupof sensor devices are within a predetermined radius; and (3)determining, by the sensor module for each sensor device of the group ofsensor devices, a respective sensor type based on respective sensoridentification information, wherein the first group of sensor devices iseach sensor device of the group of sensor devices that are of a firsttype of sensor device.

In embodiments, the senor identification information comprises, for eachsensor device, at least the following: (A) a sensor type; (B) sensorspecification information; and (C) sensor running time informationindicating a respective amount of time a respective sensor has beenoperating.

In embodiments, the sensor information further includes: (4) sensorproximity information, indicating one or more sensor devices within apredefined distance of a respective sensor device.

In embodiments, determining whether the at least one sensor device is atleast partially defective is further based at least partially onrespective sensor variance information, respective first timestamp, andrespective second timestamp.

In embodiments, the plurality of sensor devices includes at least oneof: (1) a temperature sensor; (2) a pressure sensor; (3) a torquesensor; (4) a MEMS sensor; (5) a humidity sensor; (6) an air moisturesensor; (7) an air flow sensor; (8) a dielectric soil moisture sensor;(9) an optical sensor; (10) an electro-chemical sensor; (11) anaccelerometer; (12) a gyrometer; (13) a magnetometer; (14) a proximitysensor; (15) an air bubble detector (16) a piezo film sensor; (17) anangle or position sensor; (18) a voltage sensor; (19) a current sensor;(20) a magnetic or electrical field sensor; (21) an audio sensor; (22) apH sensor; (23) a time sensor; (24) a biological sensor; (25) abiometric sensor and (26) a radiation sensor.

In embodiments, the method further comprises: (p) determining, by thecomputing device, whether a first sensor device of the first group ofsensor devices is detecting an emergency, based on at least thefollowing: (1) comparing, by the computing device, first sensor feedbackinformation associated with the first sensor device with first datarange information associated with the first sensor device, wherein thecomputing device determines that the first sensor feedback informationvaries by more than a second predetermined threshold from the first datarange information; (2) comparing, by the computing device, the firstsensor feedback information with at least second sensor feedbackinformation associated with a second sensor device of the first group ofsensor devices, wherein the computing device determines that the firstsensor feedback data is consistent with at least the second sensorfeedback data; and (3) analyzing, by the computing device a firstdetermined reliability rating associated with the first sensor device,wherein the first determined reliability rating indicates that the firstsensor device is reliable; and (q) generating, by the computing device,an emergency notification indicating that the first sensor device isdetecting an emergency event; and (r) sending the emergency notificationto the at least one administrator of the computing device.

In embodiments, the method further comprises: (p) sending, by thecomputing device to an electronic device operatively connected to thecomputing device, the notice message.

Now that embodiments of the present invention have been shown anddescribed in detail, various modifications and improvements thereon canbecome readily apparent to those skilled in the art. Accordingly, theexemplary embodiments of the present invention, as set forth above, areintended to be illustrative, not limiting. The spirit and scope of thepresent invention is to be construed broadly. Those of ordinary skill inthe art will recognize that the method and apparatus of the presentinvention described herein, and others implied have many applications;therefore, the present invention which is the subject of thisapplication is not limited by or to the representative examples and/ormethods disclosed herein, nor limited by or to the embodiments describedherein. Moreover, various other embodiments and modifications to theseexemplary embodiments may be made by those skilled in the relevant artwithout departing from the scope or spirit of these inventions.Accordingly, the inventions are not to be limited by the foregoingspecification, except as by the appended claims.

What is claimed is:
 1. A method comprising: (a) obtaining, by a sensormodule on a computer system, first sensor feedback information from eachsensor device of a first group of sensor devices of a plurality ofsensor devices operatively connected via a network to the computersystem, wherein the first sensor feedback information includes: (1)identification information associated with each respective sensor deviceproviding first sensor feedback information; (2) respective firstreadout information provided by each respective sensor device; (3)respective first metadata information associated with and provided byeach respective sensor device; and (4) respective first timestampinformation indicating a respective first time at which the respectivefirst readout information was generated by the respective sensor device;(b) generating, by the sensor module, respective second timestampinformation indicating a respective second time at which the firstsensor feedback information was obtained by the sensor module; (c)storing, in at least one memory element of memory operatively connectedto the computer system, the first sensor feedback information, includingthe second timestamp, wherein the at least one memory element furtherincludes historical sensor feedback information associated with one ormore sensor devices of the plurality of sensor devices, wherein thehistorical sensor feedback information includes: (1) identificationinformation associated with each respective sensor device of the one ormore sensor devices associated with the historical sensor feedbackinformation; (2) respective historical readout information provided byeach respective sensor device of the one or more sensor devicesassociated with the historical sensor feedback information; (3)respective historical metadata information associated with and providedby each respective sensor device of the one or more sensor devicesassociated with the historical sensor feedback information; and (4)respective historical timestamp information indicating a respectivehistorical time at which the respective historical feedback informationwas generated by the respective sensor device of the one or more sensordevices associated with the historical sensor feedback information,wherein the memory comprises: (i) a reliability rating databaseoperatively connected to the computer system, which comprises arespective historical reliability rating for each sensor device of theplurality of sensor devices, and (ii) a sensor information databasewhich includes for each sensor device of the plurality of sensordevices, sensor information including: (a) sensor identificationinformation unique to each sensor device; (b) location informationindicating a sensor location associated with each sensor device; and (C)data range information indicating a range of permissible sensor valuesfor the respective sensor device; (d) calculating, by the sensor module,for each sensor device of the first group of sensor devices, respectivesensor variance information based on at least one of the following: (1)respective first sensor feedback information; (2) respective historicalsensor feedback information; and (3) respective predictive data, whereinthe respective predictive data is generated by the computer system bypopulating a database with respective exemplary readout informationassociated with a sensor device of the first group of sensor devices;(e) calculating, by the sensor module for each sensor device of thefirst group of sensor devices, respective sensor deviation informationbased on at least one of the following: (1) respective sensor varianceinformation; and (2) first sensor feedback information; (f) storing, bythe sensor module in the at least one memory element, the respectivesensor variance information and the respective sensor deviationinformation associated with each sensor device of the first group ofsensor devices; (g) determining, by the computer system, a reliabilityrating of each sensor device of the first group of sensor devices basedon at least the following: (1) the respective sensor varianceinformation; (2) the respective sensor deviation information; and (3)the respective first sensor feedback information; (h) determining, bythe computer system, whether at least one sensor device of the firstgroup of sensor devices is out of specification by performing at leastthe following steps: (1) comparing, by the computer system, firstrespective sensor feedback information associated with the at least onesensor device with first respective data range information associatedwith the at least one other sensor device, wherein, the computer systemdetermines that the at least one sensor device is out of specificationwhen the first respective sensor feedback data varies by more than afirst predetermined threshold from the first respective data rangeinformation; (2) comparing, by the computer system, the first respectivesensor feedback information with second respective sensor feedbackinformation associated with one or more other sensor devices of thefirst group of sensor devices, wherein the computer system determinesthat the at least one sensor device is out of specification when thefirst respective sensor feedback data is inconsistent with the secondrespective sensor feedback data; and (3) determining that the at leastone sensor device is out of specification by performing the followingsteps: (A) obtaining respective reliability ratings for at least the atleast one sensor device; (B) in the event the respective reliabilityratings indicate the at least one sensor is unreliable, the at least onesensor device is determined to be out of specification; and (C) in theevent the respective reliability ratings indicate the at least onesensor is not unreliable, the at least one sensor device is notdetermined to be out of specification; (i) in the case where at leastone sensor device is determined to be out of specification, performingthe following steps: (1) generating, by the computer system, a noticemessage identifying the at least one sensor device that is out ofsequence, where the notice message includes cause information indicatingthe at least one sensor device is out of specification; and (2) sendingthe notice message to a device associated with at least one user of thecomputer system; and ( ) updating, by the sensor module, the historicalsensor feedback information with the first sensor feedback information.2. The method of claim 1, wherein in step (d), the sensor modulecalculates, for each sensor device of the first group of sensor devices,respective sensor variance information further based on: (4) calculatedresults of a Hidden Markov Model (HM) algorithm utilizing one or more ofthe following as an input: a. the respective sensor varianceinformation; b. the respective sensor deviation information; c. therespective first sensor feedback information; and d. respectivehistorical sensor feedback information.
 3. The method of claim 1,wherein in step (d), the sensor module calculates, for each sensordevice of the first group of sensor devices, respective sensor deviationinformation further based on at least one of the following: (3)respective historical sensor feedback information; (4) respectivepredictive data; and (5) calculated results of a Hidden Markov Model(HM) algorithm utilizing one or more of the following as an input: a.the respective sensor variance information; b. the respective sensordeviation information; c. the respective first sensor feedbackinformation; and d. respective historical sensor feedback information.4. The method of claim 1, wherein in step (h)(2) of comparing the firstrespective sensor feedback information with second respective sensorfeedback information further comprises comparing the first respectivesensor feedback information with one or more of the following: (A)historical sensor feedback information associated with the respectivesensor device; (B) historical sensor feedback information associatedwith the one or more other sensor devices of the first group of sensordevices; (C) predictive data associated with the respective sensordevice, wherein the respective predictive data is generated by thecomputer system utilizing respective exemplary readout informationassociated with the respective sensor; (D) predictive data associatedwith the one or more other sensor devices of the first group of sensordevices, wherein the respective predictive data is generated by thecomputer system utilizing respective exemplary readout informationassociated with the one or more other sensor devices of the first groupof sensor devices.
 5. The method of claim 1, wherein the computer systemweights respective sensor feedback information associated with one ormore sensor devices of the plurality of sensor devices that areassociated with a first respective reliability rating indicating thatone or more sensor devices are reliable more than respective sensorfeedback information associated with one or more sensor devices of theplurality of sensor devices that are associated with a second respectivereliability rating indicating that one or more sensor devices areunreliable.
 6. The method of claim 1, wherein determining whether the atleast one sensor device of the first group of sensor devices is out ofsequence further comprises: (4) obtaining, by the computer system fromthe reliability rating database, a respective historical reliabilityrating for each sensor device of the first group of sensor devices; and(5) weighting respective sensor feedback information associated withreliable sensor devices of the first group of sensor devices more thansensor feedback information associated with unreliable sensor devices ofthe first group of sensor devices.
 7. The method of claim 6, wherein, inthe event that both the respective historical reliability rating and thedetermined reliability rating associated with the at least one sensordevice indicate that the at least one sensor device is unreliable, theat least one sensor device is determined to be out of sequence.
 8. Themethod of claim 6, wherein, in the event that the respective historicalreliability rating indicates that the at least one sensor device isreliable and the determined reliability rating indicates the at leastone sensor device is unreliable, the respective sensor device isdetermined to be out of sequence.
 9. The method of claim 6, wherein, inthe event that the respective historical reliability rating indicatesthe at least one sensor device is unreliable and the determinedreliability rating indicates the at least one sensor device is reliable,the at least one sensor device is determined to be not out of sequence.10. The method of claim 1, further comprising: (k) identifying, by thecomputer system, respective first readout information associated withone or more sensors that are determined to be reliable based on thedetermined reliability rating; (1) generating, by the computer system,first machine-readable instructions to provide a first graphical userinterface, wherein the first graphical user interface presents a displayof the respective first readout information associated with the one ormore sensors that are determined to be reliable; and (m) transmitting,by the computer system to an administrator device associated with anadministrator of the computer system, the first machine-readableinstructions, wherein, upon receipt, the administrator device executesthe first machine-readable instructions such that the first graphicaluser interface is displayed on a display screen associated with theadministrator device.
 11. The method of claim 1, further comprising: (k)obtaining, by the computer system, respective sensor informationassociated with the at least one sensor device that is out of sequence;(1) generating, by the computer system, a report comprising at least thefollowing: (1) the respective sensor information; and (2) a repairmessage indicating that the at least one sensor device is out ofsequence; and (m) sending, by the computer system to the deviceassociated with the at least one user, the report.
 12. The method ofclaim 1, further comprising, (k) obtaining, by the computer system,respective sensor information associated with the at least one sensordevice that is out of sequence; (1) determining, by the computer systembased on the respective sensor information, at least one item associatedto purchase, wherein the at least one item is associated with the atleast one sensor device; (m) generating, by the computer system, apurchase order comprising at least the following: (1) the respectivesensor information associated with the at least one sensor device thatis out of sequence; and (2) the at least one item; and (n) sending, bythe computer system, the purchase order to a third party vendor.
 13. Themethod of claim 12, further comprising: (o) generating, by the computersystem, a report comprising at least the following: (1) the respectivesensor information; (2) information identifying the at least one item;and (3) a repair message indicating at least the following: (i) that theat least one sensor device is out of sequence; and (ii) the at least oneitem; and (p) sending, by the computer system to the device associatedwith the at least one user, the report.
 14. The method of claim 13,further comprising: (q) sending, by the computer system to an electronicdevice operatively connected to the computer system, the repair message.15. The method of claim 1, further comprising: (k) obtaining, by thecomputer system, respective sensor identification information associatedwith the at least one sensor device that is out of sequence; (1)storing, in a flagged sensor database of the computer system, therespective sensor identification information; (m) generating, by thecomputer system, second machine-readable instructions to: (1) flag, bythe flagged sensor database, obtained respective sensor feedback dataassociated with the sensor identification information stored in theflagged sensor database; and (2) store, in the flagged sensor database,the flagged respective sensor feedback data; and (n) executing, by thecomputer system, the second machine-readable instructions.
 16. Themethod of claim 15, further comprising: (o) generating, by the computersystem, third machine-readable instructions to: (1) ignore data receivedfrom the at least one sensor device that is out of sequence; and (2)cease transmitting data to the at least one sensor device that is out ofsequence; and (p) sending, by the computer system to each sensor deviceof a second group of sensor devices, the third machine-readableinstructions, wherein the second group of sensor devices is the firstgroup of sensor devices less the at least one sensor device, andwherein, upon receipt of the third machine-readable instructions, eachof the second group of sensor devices executes the thirdmachine-readable instructions.
 17. The method of claim 1, furthercomprising: (k) obtaining, by the computer system for each sensor deviceof the first group of sensor devices, first weather informationassociated with at least a first respective temperature at the firsttime within a predetermined radius of respective location information,wherein the sensor variance information is further based on the firstweather information, and wherein the sensor deviation information isfurther based on the first weather information.
 18. The method of claim1, wherein the first group of sensor devices selected by the sensormodule by performing the following steps: (1) determining, by the sensormodule for each sensor device of the plurality of sensor devices,respective location information; (2) selecting, by the sensor module, agroup of sensor devices of the plurality of sensor devices based on therespective location information, wherein each sensor device of the groupof sensor devices are within a predetermined radius; and (3)determining, by the sensor module for each sensor device of the group ofsensor devices, a respective sensor type based on respective sensoridentification information, wherein the first group of sensor devices iseach sensor device of the group of sensor devices that are of a firsttype of sensor device.
 19. The method of claim 1, further comprising:(k) determining, by the computer system, whether a first sensor deviceof the first group of sensor devices is detecting an emergency, based onat least the following: (1) comparing, by the computer system, firstsensor feedback information associated with the first sensor device withfirst data range information associated with the first sensor device,wherein the computer system determines that the first sensor feedbackinformation varies by more than a second predetermined threshold fromthe first data range information; (2) comparing, by the computer system,the first sensor feedback information with at least second sensorfeedback information associated with a second sensor device of the firstgroup of sensor devices, wherein the computer system determines that thefirst sensor feedback data is consistent with at least the second sensorfeedback data; and (3) analyzing, by the computer system a firstdetermined reliability rating associated with the first sensor device,wherein the first determined reliability rating indicates that the firstsensor device is reliable; and (1) generating, by the computer system,an emergency notification indicating that the first sensor device isdetecting an emergency event; and (m) sending the emergency notificationto the at least one administrator of the computer system.
 20. The methodof claim 19, further comprising: (n) sending, by the computer system toan electronic device operatively connected to emergency services, theemergency notification.